accept-encoding nomalization

압축된 Contents를 Caching 하는 NGINX Proxy의 동작 
a. Accept-Encoding을 cache key에 넣지 않아도 캐쉬 키가 달라짐

(Vary 문제 해결)
proxy_ignore_headers Vary; 설정으로 "accept-encoding: br, gzip" 요청도 br로 HIT됨
Vary 해더를 무시하지 않으면 Accept-Encoding이 조금만 달려져도 별도의 Cache Key로 판단 MISS발생
(참고 자료) https://www.fastly.com/blog/best-practices-using-vary-header

 --> proxy_ignore_headers Vary; 로 응답해더에서 Vary를 제거 시 HIT됨

# curl relay/1.json  -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:28:59 GMT
Content-Type: application/json
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: W/"65d597cc-10fa"
Cache-Control: max-age=120
Content-Encoding: gzip
X-CACHE-STATUS: HIT

# curl relay/1.json -H "accept-encoding: gzip" -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:28:53 GMT
Content-Type: application/json
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: W/"65d597cc-10fa"
Cache-Control: max-age=120
Content-Encoding: gzip
X-CACHE-STATUS: HIT

# curl relay/1.json -H "accept-encoding: br" -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:29:50 GMT
Content-Type: application/json
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: W/"65d597cc-10fa"
Cache-Control: max-age=120
Content-Encoding: gzip
X-CACHE-STATUS: HIT

 

(문제점) 맨 마지막으로 저장된 Content-Encoding: gzip 으로 HIT되어 Plain요청이나 br요청에 잘못된 gzip응답을 하게됨
(해결방법) 1. Accept-Encoding을 정규화 처리하여 Cache key 로 저장
                  2. gzip을 캐시하고, gzip 인코딩을 허용하지 않는 클라이언트에 대해 압축 해제
                  3. proxy_set_header Accept-Encoding ''; 을 설정해 모든 요청을 압축하지 않은 채 저장

(참고) 
https://dzx.fr/blog/how-to-configure-reverse-proxy-caching-with-nginx/
https://serverfault.com/questions/1005628/is-it-safe-to-use-proxy-ignore-headers-vary

Is it safe to use proxy_ignore_headers Vary?

 

2. Accept-Encoding Normalization

  set $encoding "";
  if ($http_accept_encoding ~ br) { set $encoding "br"; break; }
  if ($http_accept_encoding ~ gzip) { set $encoding "gzip"; break; }
  proxy_cache_key $host$uri$is_args$args$encoding;

 

퍼지 후 2번 테스트하면 MISS 후 HIT되고 Encoding이 정확히 일치함

# curl relay/1.json  -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:50:43 GMT
Content-Type: application/json
Content-Length: 4346
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: "65d597cc-10fa"
Cache-Control: max-age=120
X-CACHE-STATUS: HIT
Accept-Ranges: bytes

# curl relay/1.json -H "accept-encoding: gzip" -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:50:17 GMT
Content-Type: application/json
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: W/"65d597cc-10fa"
Cache-Control: max-age=120
Content-Encoding: gzip
X-CACHE-STATUS: HIT

# curl relay/1.json -H "accept-encoding: br" -I
HTTP/1.1 200 OK
Server: nginx/1.25.4
Date: Sat, 27 Apr 2024 14:50:28 GMT
Content-Type: application/json
Connection: keep-alive
Last-Modified: Wed, 21 Feb 2024 06:27:24 GMT
Vary: Accept-Encoding
ETag: W/"65d597cc-10fa"
Cache-Control: max-age=120
Content-Encoding: br
X-CACHE-STATUS: HIT