[Feature]: Refactor normalizeHref for Optimal SEO URL Handling

[coratgerl]

What problem does this feature solve?

Currently, the normalizeHref function in the rspress repository transforms URLs in a way that may not be optimal for SEO. Specifically, it converts / to /index for clean URLs and / to /index.html for non-clean URLs. This behavior can lead to suboptimal SEO performance because canonical URLs (shorter and cleaner) are generally better for search engine rankings.

Current Behavior:

Clean URLs: / -> /index
Non-clean URLs: / -> /index.html

Proposed Changes:

Refactor the normalizeHref function to always use the root URL / instead of appending index or index.html.
Consider removing the cleanUrls option as it introduces non-optimal behavior from an SEO perspective.

Benefits:

• Improved SEO by using canonical URLs.
• Simplified URL structure, making it easier for users and search engines to understand.
• Removal of redundant cleanUrls option, reducing potential confusion and misuse.

If you are ok I can create a PR for this changes

[Timeless0911]

IMO, I think we also need to sort out and unify the concatenation of version / lang / /index / cleanUrl in the url. This is a more complicated matter and may require a certain amount of refactoring.