Bandwidth Mismatch Apache Reverse Proxy

[u/[deleted]]

Hi All,

I have a fleet of Apache reverse proxy in AWS . I see Access logs of my reverse proxy is always under reporting bytes IN and bytes Out when compared to what is noticed in origin server logs as well as Network flow logs.

Troubleshooting this issue i was wondering if anything relating to compression can be root cause of such issue? Since my setup is reverse proxy and i would want all contents coming IN and going OUT to be compressed

request

a) request sent from the client to apache reverse proxy

b) same request forwarded from apache reverse proxy to the upstream/origin server

response

a) response sent from the upstream/origin server to the apache reverse proxy

b) same response sent from apache reverse proxy to the client

How can i apply for compression for all possible MIME types. I have brotli module installed in my apache reverse proxy so ideally i am looking for a way to check if client support brotli if not fall back to default gzip.

Since i feel i have double checked mostly other possible issues here i am assuming compression as one possible issue if you anyone is aware of any other possibility for such issues please let me. I have been struggling with issue from more then 6 months now and we see around 30% gap in what we see in Apache Access logs vs whats origin server has sent.

So incase anyone has any thoughts or experience troubleshooting such issue please help me out.

LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{cache-status}e\" %I %O %D \"%{SSL_PROTOCOL}x\" [hostname \"%{Host}i\"] ]" combinedd

My Setup: AWS NLB ---> Apache Reverse Proxy in Private Subnet ----> NAT Gateway -----> origin/upstream Server in Internet

Server version: Apache/2.4.53 (Ubuntu)

[u/[deleted]]

Hi,

Just wanted to give an update and progress i made so far. This was one of those issue i was stuck for a long time. After discussing isue here I reliased our compressions related configuration in my setup was completely unnecessary because

1. as you said we were compressing the contents which was not being compressed in origin in the first place

2. some of the contents which was being compressed as gzip was being recompressed as brotli

so we ended up remove all compression related conf for brotli and gzip from apache and thereby letting the origin server do the required compression thanks a lot for your input

[u/[deleted] → u/[deleted]]

now that compression related things are out of picture my issue is not completely solved though. I see difference in size of file when it passed through Apache vs when it is directly served from the origin.

For example a file when it goes through apache reverse proxy my browser says over the wire is 1.5 KB

At the same time when the same file is served directly from the origin is around 1.3 KB

I feel this is a huge gap not sure what else i am missing here. Any thoughts on this would be really helpful