0

This is a followup question

  • My intended .htaccess ruleset has overall been achieved.
  • See that file in its current form quoted below.

Remaining problem: One ruleset yet not working

  • Goal: Normalize request(\.htm|\.html) to just the beautiful request :
RewriteCond %{REQUEST_URI} ".+\.html?$"
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
RewriteRule (.*)/([^/]+)\.html?$ $1/$2 [R=301]

Observed behavior

  • Requesting /hello served the underlying /hello.html.
    • This is desired. Defined in rule 2d) DirectoryIndex features implemented in mod_rewrite.
  • Requesting /hello.html served it as is.
    • This is undesired.
    • Because serving the same file both with and without suffix means duplicate content and non-uniformity. Which I want to avoid.
    • For /hello.htm(l) I wanted an enforced HTTP 301 redirect to just /hello.

Challenge

  • The syntax for this is seemingly easy, as the excerpt above shows.
  • But the correct interplay with the other rulesets ain't. Especially with rule 2a) Serve file as requested which should still work in all other cases.

Full .htaccess

## My IA technical concept: All major things work, except one detail.
#
# 1a) Security settings.
# 1b) Explicit requests to index.php CMS router --> Served as-is.
# 2) Static page overlay into CMS namespace
# 2a) /hello — File without file extension exists. --> Served as-is.
# 2b) /hello.html - File with .html extension exists.
#  - Serve in canonical form (=without extension) so at: /hello
#  - Doesn't work yet. This is where I need help.
# 2d) /hello/index.(htm|html|php) — Folder of that name with index file exists.
#  - Note: /hello/ directory listing is explicitly forbidden in Security section (1a)
#  - DirectoryIndex rules cannot "fail gracefully", hence recreated as rules in mod_rewrite. Works fine.
# 3) CMS page exists as cached file --> Cached page gets served at beautiful URL.
# 4) /index.php — If nothing of the above matched hand over to CMS index.php (e.g. Wordpress)

## 1a) Security
### Responding as if those files don't exist
RedirectMatch 404 /\.gitignore
### Directory listing OFF
Options -Indexes
### Valid index files
DirectoryIndex index.html index.htm index.php


## 1b) If CMS router is directly requested, serve as-is
RewriteEngine On
RewriteRule ^index\.php$ - [L]


### 2a) Serve file as requested
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule . - [L]


### 2b) Except if it has a .htm or .html suffix
# - For those we will redirect to the canonical URL with the file extension stripped. --> Not yet working!
# - Serve also filenames without extensions.
#     Those gets served without a content-type (MIME type). --> Works.
#       No problem for those few rare cases as parser of most web browsers detects most important types such as HTML also without a "Content-Type:" header.

# For "/request" check if there's a corresponding "/request.html" and if serve this under "/request"

#### Code block which doesn't work
# - RegEx itself is OK as tested at: https://regex101.com/r/4rTEVk/3
# - Must be a ruleset conflict or order problem
# RewriteCond %{REQUEST_URI} ".+\.html?$"
# RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f
# RewriteRule (.*)/([^/]+)\.html?$ $1/$2 [R=301]

#### So in the meanthile I simply serve files with a .htm(l) extension as-is
RewriteCond %{DOCUMENT_ROOT}/$1.html -f
RewriteRule ^([^.]*[^/])$ $1.html [L]
RewriteCond %{DOCUMENT_ROOT}/$1.htm -f
RewriteRule ^([^.]*[^/])$ $1.htm [L]


## 2d) DirectoryIndex features implemented in mod_rewrite
# - index.(htm|html|php) are valid index files
# - /hello/index.(htm|html|php) will get served at /hello

# If a directory is requested, which is missing the trailing slash then append it
RewriteCond %{DOCUMENT_ROOT}/$1 -d
RewriteRule ^(.*[^/])$ /$1/ [R=301,L]

# Optimisation: If a directory is not requested then skip the next 3 rules
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . - [S=3]

# NB: Directories end in a trailing slash (enforced above)
RewriteCond %{DOCUMENT_ROOT}/$1/index.html -f
RewriteRule ^(.+)/$ $1/index.html [L]
RewriteCond %{DOCUMENT_ROOT}/$1/index.htm -f
RewriteRule ^(.+)/$ $1/index.htm [L]
RewriteCond %{DOCUMENT_ROOT}/$1/index.php -f
RewriteRule ^(.+)/$ $1/index.php [L]


## 3) CMS pages cached to files

# BEGIN W3TC Browser Cache
# ... Various rules which set Cache-Control headers per file type.
# ... Machine generated from Admin UI.
# END W3TC Browser Cache

# BEGIN W3TC Page Cache core
# ... Various redirections rules to cached page representations.
# ... Machine generated from Admin UI.
# END W3TC Page Cache core


## 4) Fallback to CMS
# Note that from the default Wordpress .htaccess two conditions (filesystem checks) are removed.
# - The first one that checks for a "file" is handled by multiple of our rulesets.
# - The second check for a "directory" was removed otherwise directories that do not contain a "DirectoryIndex" are not routed to the CMS.

# BEGIN WordPress
# The directives (lines) between "BEGIN WordPress" and "END WordPress" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

2 Answers 2

0

Preliminary answer: 2b) Except if it has a .htm or .html suffix is never reached due to the earlier 2a) Serve file as requested being to generously matching any file!

  • So the solution is narrow down 2a) to effectively be Serve filename.ext as requested if it's indeed a file and not ending in .htm(l).
  • Full answer as soon as worked out and tested.

→ Update: Approach was not sufficient by itself! ↓ See comments. ↓

6
  • That approach alone was not enough. This still ended in an endless loop which Apache / mod_rewrite end at a certain point by force. I needed to include loop breaking mechanisms myself. See the other answer.
    – porg
    Feb 16 at 23:59
  • I think you could simply delete this one. Feb 18 at 10:36
  • I would like to keep this my preliminary answer and its shortcoming (endless loop!) to document how to NOT do it, for which symptoms people possibly search for. I hope this is in the spirit of StackExchange. If this is against best practise, then please tell me, and I delete it.
    – porg
    Feb 18 at 12:53
  • In my experience people don't read answers very carefully and could try this as a solution. Feb 18 at 13:20
  • @EsaJokinen I will delete it then. I send out this comment. Then wait 1 day to be sure the notitifcation reached you. Then will delete it.
    – porg
    Feb 18 at 17:27
0

My IA / URL scheme: Works completely now

1a. Security settings.

1b. Explicit requests to /index.php CMS router --> Served as-is.

  1. Static page overlay into CMS namespace

    • 2a) /hello — File without file extension exists. --> Served as-is without Content-Type. --> Default behavior. No rule needed.
    • 2b) /hello.html - File with .html extension exists. --> Serve HTTP 301 at /hello (= without extension = canonical form) with contents of /hello.html
    • 2c) /hello requested and /hello doesn't exist but /hello.html exists --> Serve HTTP 200 at /hello with contents of /hello.html
    • 2d) /hello/index.(htm|html|php) — Folder of that name with index file exists.
      • Note: /hello/ directory listing is explicitly forbidden in Security section (1a)
      • DirectoryIndex rules cannot "fail gracefully", hence recreated as rules in mod_rewrite. Works fine.
  2. CMS page exists as cached file --> Cached page gets served at beautiful URL.

  3. /index.php — If nothing of the above matched hand over to CMS index.php (e.g. Wordpress)

The missing rules 2b and 3c have been achieved like this

RewriteEngine On

# ...

# Rule 2b: Redirect requests for HTML files to their canonical URL without suffix
RewriteCond %{THE_REQUEST} \s/([^/]+)\.html [NC]
RewriteRule ^ /%1 [R=301,L]

# Rule 2c: Serve HTML files at their canonical URL without suffix
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule ^([^/]+)/?$ $1.html [L]

#...

Explanation of all the directives, their flags, and how an endless loop is prevented

Rule 2b: Redirect requests for HTML files to their canonical URL without suffix

RewriteCond %{THE_REQUEST} \s/([^/]+)\.html [NC]
RewriteRule ^ /%1 [R=301,L]
  • RewriteEngine On: This directive enables the rewriting engine for this .htaccess file, allowing URL rewriting.
  • RewriteCond %{THE_REQUEST} \s/([^/]+)\.html [NC]: This condition checks if the current request includes a URL with the .html suffix. The %{THE_REQUEST} variable contains the full HTTP request line sent by the browser to the server, and \s/([^/]+)\.html is a regular expression that matches requests for HTML files. The [NC] flag makes the match case-insensitive.
  • RewriteRule ^ /%1 [R=301,L]: This rule matches any URL (^) and redirects it to the URL without the .html suffix (/%1). The %1 in the substitution refers to the captured group from the previous RewriteCond. The flags [R=301,L] indicate a permanent (301) redirect and that this is the last rule to be applied for this request.
  • The loop prevention mechanism in Rule 2b is implicit. Since this rule is intended for redirection, there's no need for explicit loop prevention measures. The [L] flag ensures that this rule is the last one processed for a given request, preventing subsequent rules from causing a loop.

Rule 2c - Serve HTML files at their canonical URL without suffix

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME}.html -f
RewriteRule ^([^/]+)/?$ $1.html [L]
  • This rule set serves HTML files without the .html suffix.
  • RewriteCond %{REQUEST_FILENAME} !-f: This condition checks if the requested URL does not directly map to an existing file in the file system. If the file does exist (!-f), this condition is not met, and the rule is skipped. This condition helps prevent a loop by ensuring that the rule is only applied when the requested URL does not correspond to an actual file.
  • RewriteCond %{REQUEST_FILENAME}.html -f: This condition checks if appending .html to the requested filename results in an existing file in the file system. This condition is crucial for loop prevention because it ensures that the rewrite rule is only applied if the corresponding HTML file exists. If the HTML file does not exist, the rule will not match, preventing a loop.
  • RewriteRule ^([^/]+)/?$ $1.html [L]: This rule captures the part of the URL after the domain name using the pattern ^([^/]+)/?$ and appends .html to it. However, because of the preceding conditions, this rule is only applied if the requested file does not exist (!-f) and the corresponding HTML file exists (%{REQUEST_FILENAME}.html -f). This prevents a loop where the server continuously appends .html to the URL, as it only applies the rule when necessary.
  • The loop prevention mechanism is implemented through the use of conditions (RewriteCond) in Rule 2c. These conditions ensure that the rewrite rule is only applied when necessary, preventing potential infinite loops. Additionally, the [L] flag in Rule 2a ensures that redirection occurs only once per request, preventing a loop in the redirection process.

Example Requests and how they get processed by above rules

  1. /sand.html

    • The request is processed according to Rule 2b. The HTML suffix is removed, and a 301 redirect is issued to the client to the canonical URL without the suffix /sand.
    • Then, the rewritten URL /sand is processed according to Rule 2c. The server internally rewrites the URL to serve the content from /sand.html.
    • Finally, the content of /sand.html is served.
  2. /sand

    • The request is processed according to Rule 2c directly because the file /sand.html exists.
    • The server internally rewrites the URL to serve the content from /sand.html.
    • The content of /sand.html is served.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .