What does a textbook look like while you are writing it?
Before you spend a lot of time on formatting, contact a publisher or two and see if they want to work with you and publish your book. If they are excited enough then they will sign you up. You normally talk to an acquisitions editor at this stage.
But each publisher may have different preferred ways of working with your manuscript and it can affect how you work and the tools you use.
For the content of the book, work with whatever tools you prefer, but as you get closer to publication, the publisher will want its own formats.
Normally, for a textbook, you will be assigned an editor (in the sense of one who edits - a content editor). They will make suggestions about your wording and such. You will need to work with this person, using some mutually acceptable set of tools. It might be Word or something else. If LaTex is required, they might do that formatting themselves, and if not, will probably provide a required template for you to use. Even if they prefer Word, they will most likely have a template for you. Perhaps a required template.
You will have only limited control over placement of figures and other such things, and only limited control over layout. But the publisher will somehow prepare the manuscript for printing using its own preferred layouts: margins and such. They may even want you to use, or avoid, certain fonts.
I think that very few publishers will just print a photocopy of your manuscript. There is some processing done before production. So, don't obsess over formatting until you are working with a publisher.
Some publishers will want separate files for any figures (or even for chapters). High resolution images. Others will prefer to create the image files themselves. You may need permissions for any figures (or photos) that you don't create yourself. If the permissions need to be paid for then that may be your responsibility, or you may be given a budget by the publisher for that.
Once the manuscript is reasonably complete the acquisitions editor (or a representative) will probably send your manuscript out for review to a few people they trust. You will get suggestions for improvement of the text, possibly a lot. Especially for a first time author. Your book could get rejected at this point, but I think that is very unlikely. But you will be expected to respond appropriately to all suggestions, usually by modifying the text. This stage can be maddening for a first time author and there may be several rounds of review.
Once they have a printable version, they will send you galley proofs for your approval. These might just be pdf files and you can usually make minor edits, but a lot of money has been spent at this point, so too many changes will be problematic. But your prior work with the content editor should make this moot.
They will try to accommodate you, but you need to work with them as well.
I'm not going to recommend a particular program to use, but instead suggest considerations:
For any project that goes beyond 20 pages, it is no longer practical to change the formatting of things by hand. As a consequence you will want to use a workflow that (i) allows you to define styles for everything: headings, margins, placement of figures, fonts, etc; if you change your mind later on on how a second-level heading should be formatted, you only have to change that in one place, because it's going to be impossible to do consistently in a hundred different places if you use mark-up by hand. (ii) a workflow that allows using the same style file for each separate document that defines the book; so, if you use Word, and choose to have one file per chapter, then the style needs to be defined in an external style file that can be separately edited, rather than defined in each chapter's Word file.
Make cross-referencing easy. You will want to reference sections in other chapters and other chapter's figures. If you keep each chapter in a separate file, this is going to be difficult because you can no longer reference a tag -- and writing "see Section 5.5" is sure to become wrong very quickly if you decide to add another section before that. It must be "see Section < reference a tag>" where your word processor automatically inserts the section number that corresponds to the tag. That's most easily done if you keep everything in the same document, which also guarantees that your page numbering is always correct. I don't know MS Word well enough to know whether that requires you to keep everything in one file; in LaTeX, you can keep each chapter in one file and then just have one master file that "\include"s the individual chapters.
In your choice of word processor, anticipate what publishers in your area will likely accept. I know that you could write a book in troff, but you're unlikely going to make publishers happy with that choice.
Anticipate working with others on the project. A book is a big undertaking -- there may be areas where you want feedback from others, or simply enjoy working with a co-author. Choose a workflow that allows you to do that. A particularly bad way of collaborating, in my view, is to send files around by email. You are sure to cross paths at one point, and only one person is allowed to make changes to the file at any given time -- and it's not always clear who that is. Instead, consider a workflow that allows multiple people to make changes at the same time. An example is google documents, but you can also use github with formats that allows merging multiple people's changes to the same file (in particular, for latex). If you configure dropbox right, then that can also work: You need to make sure that the person who has a file open in a word processor locks the file so that others can't edit it at the same time.
My personal system of choice is to use LaTeX, and either share the files with my collaborators via github (if version control is useful) or via overleaf (if version control is not important). But I understand that LaTeX is not for everyone. For things that don't involve a lot of formulas, I've also happily used google documents.
While I'm sure that there are many ways to do this, the short answer is that if you contact a publisher about whether they would be interested in publishing a textbook based on your text, they will let you know how they would like the text to be delivered. I expect that they would tell you to use either Microsoft Word or LaTeX.
I personally have experience with writing a Springer textbook on computational physics together with five other co-authors. As the book is quite mathematical and every author was comfortable with LaTeX, the choice to use Springer's LaTeX template to write our book was an obvious one. With LaTeX, you can focus on content and let it handle the layout for you (helped by a little bit of markup from you). In the end, your publisher will take the the text and reformat the layout according to their standards. Still, the markup that you provide is very useful as it will let them know how you want the book to be — especially if you used a template from the publisher. For our book, the final layout was almost identical to the one we had while writing it.
You will also want to have some version control system for backing up older versions of your book, and perhaps also for collaborating with other authors. We used git for our book, though I must say that it's not easy to learn for non-programmers. These days, the online LaTeX editor Overleaf is a much simpler and more natural choice, as it handles version control and multiple authors very smoothly. With Word, I don't know of any half-decent version control system unless you also have access to Sharepoint, so you may be stuck with having to back up older versions manually.