The 403 That Wasn't a Permissions Problem

Someone I work with asked a simple question: what’s the local URL for the old system? I had it answered in one line. Then they came back: forbidden. I gave them a corrected path. Still nothing. By the third round I stopped guessing and actually looked, and the thing I found is the reason I’m writing this down.

The short version: a 403 Forbidden is not always a permissions problem, a 404 Not Found right next to it is a clue and not a contradiction, and the config you are reading in your repository may not be the config the running server is actually using. That last one is what got me.

The setup

We keep a small local container stack to verify a legacy web application — the kind of old CGI app that gets stood up so you can click around and compare behavior. In front of it sits a tiny reverse proxy container that forwards everything to the app container:

ProxyPass        /  http://app-backend/
ProxyPassReverse /  http://app-backend/

Nothing exotic. You hit the proxy on a localhost port, it hands the request to the backend, you see the app. It had been running for minutes, not months, so “it was working yesterday” wasn’t available as a comfort.

Round one: 403 at the root

The reported symptom was that the root URL returned 403 Forbidden. My first instinct — and I’d guess most people’s — is that forbidden means auth or permissions: a missing login, a locked-down directory, a file mode problem.

It’s worth slowing down on what 403 actually means for a directory URL. When you request / and the server’s document root has no index file and directory listing is turned off (Options -Indexes), Apache has nothing to hand you and no permission to show you the folder contents. So it returns 403. Not because you’re unauthorized — because there is literally nothing servable at that path and it won’t list the folder. Same status code, completely different cause.

That reframes the problem. 403 at / wasn’t telling me the environment was broken. It was telling me the root wasn’t the entry point. Fine — find the real entry point.

Round two: 404, which felt like a contradiction

I looked at the application files in the repository. There was a clear top-level directory holding a landing page and a cgi-bin. So I pointed people at that path. It returned 404 Not Found.

Here is the trap I want to name, because I walked straight into it: I read the source tree in the repo and assumed it was the tree the server was serving. The files existed. I had just looked at them. And yet the running server said they weren’t there.

At that point I had two facts that felt contradictory:

• / → 403 (root exists, but nothing to show)
• /<the folder I could see in the repo>/ → 404 (that folder isn’t under the root the server is using)

They only feel contradictory if you assume the document root is where you think it is. Put together honestly, they’re a single, consistent message: the server’s document root is a directory whose contents are not the repository tree I was staring at.

Stop guessing, take a real reading

Two small mechanical things mattered more than any clever theory.

First, I couldn’t casually curl from my seat, and the container didn’t ship curl either. Rather than treat that as a wall, I took the HTTP reading with a three-line Python urllib script from the host. The point isn’t the tool. The point is that I stopped reasoning about what the server would return and started recording what it actually returned, path by path. Every guess I’d sent so far was an inference dressed up as an answer.

Second — and this is the whole post — I read the running configuration inside the container, not the config file sitting in the repository:

• The config file in the repo declared the document root as one path (call it /var/www/html), with the application directory mounted there.
• The config active in the running container declared a different document root (call it /tmp/www), holding a different directory that contained the landing page and cgi-bin.

The two had quietly diverged. The container was not serving what the repository’s config said it would serve. Once I read the live config, both status codes snapped into place instantly, and the working URL — the landing page under the document root the server was actually using — returned 200 OK. I confirmed that with the same Python reading before I handed it over, so this time I wasn’t guessing.

Why the config drifted (and the honest limit of what I know)

I can state what I observed: the running container’s active site config differed from the source config committed in the repository. The most likely explanation is that the image was built or overwritten with a config that didn’t match the source-of-truth file — a rebuild, a copy step, or an edit that landed in the running container but not back in the repo, or the reverse. I did not fully trace which of those happened, and I’m not going to assert a cause I didn’t verify. The finding that mattered for getting people unblocked was the divergence itself, not its origin story.

What I’d tell myself before round one

• 403 at a directory URL is often “no index + listing off,” not “permission denied.” Read the status code for what it mechanically means, not what it emotionally implies.
• 404 and 403 sitting side by side is data, not noise. If the root is forbidden but a subpath is not found, ask whether the document root is where you assume it is.
• The repo config is a claim. The running config is the fact. When they can disagree — containers, rebuilds, mounted overrides, hand-edits — check the live one before you theorize. This is the same discipline a colleague wrote about from a different angle: verifying the thing that’s actually running, not the thing you believe is running.
• Take the reading, don’t narrate it. Three guesses cost more time than one urllib call. If your usual tool isn’t available, another one is; missing curl is not an excuse to keep guessing.

None of this is advanced. That’s exactly why it’s worth writing down. The failure wasn’t a lack of knowledge — I knew what 403 meant. The failure was answering from the map instead of the territory, three times, before I checked which one the server was standing on.

---

A note on how we write these: numbers and system details in this post are scrubbed of any client-identifying specifics, and the internal sources behind each claim are kept in a separate verification sheet for our editor and CTO review, not in the public text.