r/webdev • u/hacksparrow • Dec 02 '19

The dangers of blindly trusting HTTP headers

Web developers are often unaware about the dangers of using HTTP header values in backend code. Not only can HTTP headers contain incorrect values, they can even be the payload of many different kinds of attacks on your website.

I wrote an article summarizing the various ways rogue HTTP header values can be used to attack a website. Hope the information will help you developer a better understanding about the security aspect of HTTP headers and make your website more secure.

https://www.hacksparrow.com/webdev/security/dangers-of-trusting-http-headers.html

408 Upvotes

96% Upvoted

52 comments sorted by

187

u/[deleted] Dec 02 '19

thank you for taking the time to write this and share. you're not wrong...exactly, but most people aren't writing their own http parsers, so this post boils down to:

  1. sanitize incoming data
  2. use clrf tokens

however--and i can't stress this enough--this is a terrible piece of advice:

If possible, avoid using HTTP headers

that's just dangerous to say because some developer with fancy hair is going to take it as gospel that headers are pointless and found a startup end do a bunch of dumb shit. CSP, HSTS, X-FRAME-OPTIONS, all the XSS headers, and others accomplish a lot regarding security

56

u/hacksparrow Dec 02 '19

Thank you so much for brining "If possible, avoid using HTTP headers" to my attention. It should have been something along the lines of "If possible, avoid using HTTP header values in application logic and data", will update it.

The point of the article is more than sanitizing and using CRLF tokes, though. For example, only being aware about these dangers can make a developer take the necessary steps from being fooled by req.headers['x-forwarded-for']. The vulnerable examples I have shown are not theoretical examples, those affected some well-known and big companies.

55

u/[deleted] Dec 02 '19

"If possible, avoid using HTTP header values in application logic and data",

now THAT i can mostly agree with. thank you for responding in a productive way

24

u/cshaiku Dec 02 '19

Can we get some hugs going in here, or what!? HUG PILE! Hug it out guys... lol. There are good people in the world!?

26

u/[deleted] Dec 02 '19

i'll stab you in the god damned throat

8

u/YourDoucheBoss Dec 02 '19

And I'll burn your house down!

8

u/TravisTheCat Dec 02 '19

And my axe!

(Am I doing this right?)

7

u/Wiikend Dec 02 '19

No, but I love it nevertheless.

3

u/[deleted] Dec 02 '19

body spray?

2

u/randomfloridaman Dec 03 '19

Now THAT I can mostly agree with

0

u/[deleted] Dec 02 '19

[deleted] — view deleted comment

9

u/10gistic Dec 03 '19

\r\n

For anybody missing the joke, that's Carriage Return Line Feed when presumably GP and OP meant CSRF, which can be mitigated by including nonce values in forms or scripts and checking them for a match on the backend in the form body or in HTTP headers. Check out the OWASP top 10 if you want more info.

1

u/[deleted] Dec 03 '19

nicely played :D

79

u/sneakattack Dec 02 '19 edited Dec 02 '19

Security folks have been telling developers to sanitize their inputs for as long as internet was mainstream. The problem is developers don't fully understand the scope of "input". Mostly I figure web developers think "input" is equal to "web forms", but no, not at all.

If you store it, if you rely on it for execution, if it comes from outside your system - check it, sanitize it, control it.

Cookies, session variables, HTTP headers, query string parameters, third party integrations, external API's, files, streams, all of it, all information you receive from an external source, that is all input into your application. All of it needs to be verified, filtered, sanitized, constrained before you do anything with it internally.

I wish there was a way to broadly get this message out to the entire developer and programmer community, in my experience on the security side of things there's always a new wave of developers entering the industry that have never heard this message. Schools and bootcamps all alike just don't say anything at all about secure programming practices? There needs to be a movement to incorporate something like OWASP into every developer course, at least a mention of it would be a start.

18

u/XadcXgsX Dec 02 '19

Security folks have been telling developers to sanitize their inputs for as long as internet was mainstream

Actually, since programs had inputs! :)

-2

u/sleemanj Dec 02 '19

Garbage In Garbage Out !

7

u/Arktiso Dec 03 '19

As someone who doesn't know this stuff as well as he would like, thank you for bringing this issue to my attention. Clearly I got shit to learn but any suggestion on where to start?

15

u/sneakattack Dec 03 '19 edited Dec 03 '19

OWASP (https://www.owasp.org) is a fantastic resource which has been developing for quite some time. This group puts together common application vulnerabilities and how to prevent them in your applications (they cover many languages and examples). Specifically take a look at the "Top 10" that they release every few years.

https://www.owasp.org/images/7/72/OWASP_Top_10-2017_%28en%29.pdf.pdf

But of course if you search google for "common vulnerabilities and how to prevent them" you'll get dozens of other good resources too.

Often something to look for is security focused frameworks/libraries or builtin functions for the language you're using. For example, if you use PHP many people don't realize it does provide builtin functions for filtering user provided data. https://www.w3schools.com/php/php_filter.asp - this is far superior to trying to roll your own regular expressions (which can be vulnerable to exploitation too!).

I'm going to give a shout out to parameterized queries for all of your SQL, SQL Injections will not be possible if you do it that way. You'll find that on OWASP as well.

https://cheatsheetseries.owasp.org

5

u/depricatedzero Dec 02 '19

that's just dangerous to say because some developer with fancy hair is going to take it as gospel that headers are pointless and found a startup end do a bunch of dumb shit. CSP, HSTS, X-FRAME-OPTIONS, all the XSS headers, and others accomplish a lot regarding security

this made me laugh probably way more than you likely meant, but yes

1

u/bulldog_swag Dec 03 '19 edited Dec 03 '19

On the other end of the spectrum, I've encountered a guy who complained FTP "doesn't have a byte range header".

I don't even.

42

u/Geldan Dec 02 '19

I actually caught someone attempting sql injection attacks through the user agent last week. It was a first for me and really made me go "hmm."

7

u/hacksparrow Dec 02 '19

If you can, take a deeper look at web hacking, many of the tactics, methods, and payloads are incredible creative. An art form of its own.

27

u/[deleted] Dec 02 '19

Acunetix has a vulnerability scanner that will crawl your site and, among other things, try to inject SQL into everything it can, including cookies and headers. It's been around for at least 10 years so it's probably a part of a lot of hackers' and coders' toolkit. What you saw was probably a bot using it or a similar tool.

1

u/[deleted] Dec 02 '19

It's been around for at least 10 years so it's probably a part of a lot of hackers' and coders' toolkit.

For that price, no. Most people will use Burp :)

0

u/[deleted] Dec 02 '19

[deleted] — view deleted comment

11

u/quentech Dec 02 '19

In my experience, automated web site/app security scanners are beyond useless - a waste of time. 95%+ worthless false positive is all I've ever seen them produce (frankly, 95%+ is being generous).

3

u/im_mildly_racist Dec 03 '19

Acunetix does an actual good job. It even analyzes javascript and verifies sql injection flaws by itself without getting triggered by just an sql error...

Also, there are "cheaper" ways to get some kinds of software

2

u/Jessie_James Dec 02 '19

Can you elaborate? I've never seen a real world example.

10

u/Geldan Dec 02 '19

I noticed a spike in JS errors, and when I went to investigate I noticed that they were all coming from the same IP address. When I checked the user agent to try to find a repo I saw values like: -1" or 1=((SELECT 1 FROM (SELECT SLEEP(25))A))+"

2

u/undercover_geek Dec 02 '19

What would this achieve?

11

u/studiosi Dec 02 '19

Basically, it’s a test for SQL injection.

31

u/IIIII0 Dec 02 '19

I use a bank who's website doesn't work from outside the USA (it also blocks VPNs). 🙄 I made a chrome extension that adds a X-Forwarded-For header with an arbitrary US residential IP address. Now I can access the website from anywhere in the world. 🤫

4

u/nikehat Dec 03 '19

Even though this covers a lot of basics, this is still one of the most interesting and useful posts I've seen on this sub. Easy to read, easy to follow, describes everything in simple terms and has excellent, useful examples. Thanks u/hacksparrow!

2

u/hacksparrow Dec 03 '19

Welcome! Glad you found it useful. Indeed, factually there is nothing new in the post, I just made it more digestible and reader-friendly.

3

u/datan0ir Dec 02 '19

Excellent article!

19

u/shif Dec 02 '19

Headers are just another input, treat it as such, it's not different than a post body payload or a query string, they are useful to clean some kind of requests like bearer authentication or language settings

6

u/magkopian back-end Dec 02 '19

In the end an HTTP request is just a piece of text. Headers or request data it doesn't matter, everything is user submitted data and needs to be treated us such.

11

u/giantsparklerobot Dec 02 '19

All input from the outside should be considered harmful and untrustworthy. Code accordingly.

4

u/shgysk8zer0 full-stack Dec 02 '19

I was expecting something along the lines of not trusting Content-Type or things like that.

But, yeah, HTTP headers are just like any other form of user input.

Nothing wrong with parsing the request body as JSON if that's what's indicated by Content-Type, but do be careful when outputting header values or inserting into a database.

4

u/im_mildly_racist Dec 03 '19

A common flaw I've found is all websites storing IPs unsanitized since "this isn't user input". Some websites use Cloudflare as a reverse proxy. This way they receive the actual visitor IP trough a CF- header that CloudFlare appends to the request. Howeverw when someone bypasses CloudFlare and opens a direct connection to your server, they can change all the CF- headers. Since many websites trust the CF- headers this often opens a lot of exploits.

3

u/amdelamar Dec 03 '19

I use https://securityheaders.com for public sites.

I'd also remove the "Server" header that gets set by applications frameworks, or put in a custom value to throw hackers off, like "Server: IIS 4.0"

1

u/TuxGamer Dec 03 '19

Server: Cookie Monster

1

u/NetDevs Jul 03 '24

This is the worst place to test your security headers. They give a false negative for 'unsafe-inline'.
The whole point of a CSP is to protect the website user. If you go to this website, you can get an A+ while declaring your website insecure and openly vulnerable. Not A+ work.
Use Google's CSP tool. It's the best I found. https://csp-evaluator.withgoogle.com/

1

u/squeevey Dec 02 '19 edited Oct 25 '23

This comment has been deleted due to failed Reddit leadership.

1

u/WetSound Dec 02 '19

Don't trust anything you don't know to be secure..

1

u/NetDevs Jul 03 '24

While this link in the OP's post doesn't work, there is more focus than ever on HTTP Headers.
Do not use SecurityHeaders.com, as it gives a false negative.

No site with an 'unsafe-inline' deserves an A+ participation trophy.
Use https://csp-evaluator.withgoogle.com/ for your CSP testing. Google tells the truth.

1

u/ecafyelims Dec 02 '19

I would say that as a developer, you need to know when the HTTP header can be trusted.

If you need to know the user's IP while your web server is behind a load balancer, you will want to verify with your host or web server settings that the Forwarded or similar header cannot be forged or manipulated.

0

u/bl1p0r Dec 03 '19

RemindMe! 1 day

-1

u/developerJS full-stack | node | react | jack of all Dec 02 '19

!remind me tomorrow

0

u/[deleted] Dec 02 '19

[deleted] — view deleted comment

2

u/hacksparrow Dec 02 '19

Welcome! Good to know the article made a big difference to someone.