Why buy high end hardware firewalls?
Solution 1:
It's just a matter of scale. The thousands-of-dollars firewalls have features & capacity allowing them to scale & be managed globally. A myriad of features that anyone not using them would have quite a bit of research to do before they (we) could appreciate their individual merits.
Your typical home router doesn't really need to be able to handle an officeful of devices or multiple ISP connections, so it's cheaper. Both in the number/type of interfaces, and the hardware capacity (RAM, etc). The office firewall also may need some QoS, and you might want it to be able to make a VPN connection to a remote office. You'll want slightly better logging for that small office than you'd need for the home firewall, as well.
Keep scaling that up until you need to handle a few hundred or thousand users/devices per site, connect to dozens/hundreds of other firewalls the company has globally, and manage it all with a small team in one location.
(I forgot to mention IOS updates, support contracts, hardware warranties - and there are probably a few dozen other considerations that I don't even know about...but you get the idea)
Solution 2:
Typically, along with the hardware firewall you get a recurring yearly maintenance fee and the promise of a future date when "hardware support" won't be available anymore and you'll have to forklift the gear out and replace it (ala the Cisco PIX to ASA transition). You also get stuck with a relationship with a single vendor. Try and get software updates for your Cisco PIX 515E from some other Cisco Systems, for example.
You can probably tell that I'm fairly negative about purpose-built firewall hardware.
Free and open source (FOSS) operating systems power some well-known "hardware" firewall devices and aren't unproven technology by any stretch. You can buy software support agreements for FOSS from many different parties. You can purchase whatever hardware you want with whatever spares / service agreement you choose.
If you're really pushing a lot of bits around then, perhaps, a purpose-built hardware firewall device would be necessary. FOSS can cover you in a lot of situations, though, and give you tremendous flexibility, performance, and total cost of ownership.
Solution 3:
You've had some good answers already talking about technical stuff and support. All important things.
Let me introduce another thing to consider: Your time to create, configure and support a "roll your own" hardware firewall internally is an investment for your employer. Like all things, the business has to decide if that investment is worth it.
What you/your manager need to consider is where your time is best spent. The question of whether or not "rolling your own" is worthwhile might change completely if you're a specialist network security person and/or your employer has specialist firewall requirements that aren't easy to setup in an off shelf product compared to someone who has lots of duties to consider besides network security and whose needs can easily be met by plugging in a network appliance.
Not just in this specific case but in general, there's been a few times I've purchased a solution "off the shelf" or hired in some consultancy for something I'm quite capable of doing myself because my employer would rather my time was spent elsewhere. This can be quite a common case, especially if you're facing a deadline and saving time is more important than saving money.
And don't discount the ability to "blame someone else" - when you've traced a major outage to a bug in the firewall at 3am in the morning it's very nice to be able to speak to the vendor and say "I don't care if its software or hardware, its your problem either way".
Solution 4:
how will your homebrew firewall handle in-service hardware maintenance?
how will your homebrew firewall hold up when you get to 40+Gbps throughput?
how will your homebrew firewall segment permissions for administrators in different business units, such that they can only manage their own parts of the rule base?
how will you manage your rulebase when you have 15,000+ rules?
who is backing you up when it goes in the ditch?
how will it hold up to a common criteria audit.
by the way, $100k is not anywhere near "high end" for firewalls. another zero would get you there. and it's really a drop in the bucket for the resources that they protect
Solution 5:
Clearly there is no one-size-fits-all answer to this question, so I'll describe what I've done and why.
To set the picture: We're a fairly small business with around 25 office staff and perhaps the same number on the production floor. Our primary business is as specialised printers who at one time enjoyed a monopoly but are now fighting an increasing amount of opposition from cheap imports, mostly from China. This means that while we would love Rolls Royce level service and hardware we generally have to settle for something more along Volkswagon levels.
In our situation the cost of something like Cisco or similar just couldn't be justified, especially as I have no experience with it (I'm a one-man IT "department"). Also, the expensive commercial units offer no true benefit to us.
After looking at what the company had and what they needed I chose to use an old PC and install Smoothwall Express, partly because I had been using that product for a number of years and was already confident and comfortable with it. This does of course mean there is no external support for the firewall, which carries a degree of risk, but it's a risk the company is comfortable with. I'll just add that as a firewall Smoothwall is as good as I've seen for our kind of scale but it may not necessarily be the best choice for a much larger organisation.
That solution works for us. It may or may not work for you. Only you can make that decision.