Best way of creating large bit arrays in Lua

You can store the data in a string and manipulate it with the string library and Lua BitOp

Lua5.2's built-in bit32 library is preferred if available.

If you want to read 1 MB into memory, you won't end up with 250 kB...

If you read the file into a Lua string, you end up with 1 MB, as Lua strings are just 8-bit clean bytes.

After that, you can process the data according its structure, using the perhaps the struct library.

You can store a big blob in Lua string, it will work with any binary data. Now the question is what you want to do with the data. In any way, you can use string.byte to extract any individual byte, and use Lua's bit32 library to get down to bits. (For Lua 5.1 and older, you'll either have to write your own C routines, or use third party package.)