Intermittent error through IIS reverse proxy

I have a very intermittent error when accessing my FlightFeeder (PiAware) via an IIS reverse proxy. It appears some request give me a 502.3. It’s not consistent and most of the time it works fine. I haven’t been able to track down the root cause.

It typically stems from a request to the following files (not exhaustive list):
skyaware/data/history_xx.json (multiple history JSON files with query strings)
skyaware/db/AB8.json
skyaware/?fbclid=xxx
skyaware/flags-tiny/xxx.png
skyaware/data/aircraft.json
skyaware/data-978/aircraft.json

I have the trace logs from the IIS reverse proxy server but without the logs from the PiAware, I can’t really correlate the two or track down the issue. IIS just says: “The server returned an invalid or unrecognized response”. Looking for any insight or tweaks I may have to do to get this working 100%.

Are there any tweaks I need to due on the PiAware side of things?

I believe I resolved this by tweaking a few IIS/ARR settings and rewrite rules. Specifically, rewriting the following HTTP variables, which are not allowed by default

                    <serverVariables>
                        <set name="HTTP_UPGRADE" value="{HTTP_UPGRADE}" />
                        <set name="HTTP_CONNECTION" value="{HTTP_CONNECTION}" />
                        <set name="HTTP_ACCEPT_ENCODING" value="&quot;&quot;" />
                    </serverVariables>

This allows IIS/ARR to proxy WebSockets, which is believe is required for PiAware to function properly, and terminates compression between IIS/ARR and the PiAware which can be the source of decompression or payload errors. Proxing “HTTP_CONNECTION” may not be required as IIS/ARR has some mechanism to manage connection lifetimes itself or it may need to be forced to “close” to disable keep-alive connections if the PiAware silently drops the connection after a long period of time but so far I haven’t had any issues with the configuration above.

In order to rewrite these HTTP variables, you must register them in the “Allowed Server Variables” list first. Then, you may add them to the rewrite rules.

I also tweaked these two ARR proxy settings:

Setting the timeout to 300 ensures the connection stays open longer if no data is sent by the PiAware. The default in IIS/ARR is 120.

The buffer threshold set to 0 is required so that IIS/ARR does not wait for data to pile and instead proxies it immediately. This prevents timeouts between IIS/ARR and the PiAware. The default in IIS/ARR is 256.

Still testing things but so far, so good.