Generate xlsx file from Apex

You're not going to be able to use XLSX at all. It's based on the Office Open XML standards from Microsoft. Basically, it's a set of XML files contained in a ZIP file, of which we have no native means of creating in Apex Code. Of course, you could build the constituent parts and then ship the data off to Heroku, AWS, etc, and have it finish packing up the file for you, or even use jszip to render the file in the browser, but the point is, it won't be a pure native solution.

As for using the "XLS" format, what you basically want to do is create a large XML file that conforms to Microsoft Office XML formatting schemes. This is actually how Salesforce "tricks" Excel into reading report exports when you use the "Export to Excel" report feature. You'll probably want to read the XML Spreadsheet Reference on how to build a proper spreadsheet. Alternatively, download a report from Salesforce in Excel format, and open it up in Notepad to see how they do it; it's basically a bare-bones implementation that's just good enough for Excel to figure out what to do with it.

I know the question has already been answered but to anyone who stumbles upon here looking for other options, I found a solution that generates an actual .xlsx file using only Apex. Take note though that the process is complicated and would take a bit research and trial and error on the side of Office Open XML to pull off.

If you already read the answer that @sfdcfox gave, you know that the biggest hurdle for this to work is Apex having no native means of working with ZIP files. Luckily someone implemented classes that would make it possible. That project is called Zippex and its made by Pedro Dal Col and Pliny Smith.

With that issue out of the way, we are now able to open up a .xlsx file and extract its contents; edit the content and fill it up with data from our Salesforce org; and then repackaged it again as a .xlsx file. All of that using only Apex. And that's generally the steps needed to generate a .xlsx file using Apex for my approach. If you like a further detailed guide for this and a working example, here is a repository I created that will be able to help you: https://github.com/RaymarMonte/apex-create-xlsx