What happens when we bring an electron and a proton together?

Part 1: Conceptual/physical intuition

Since there is an electrostatic attraction between the 2 particles, then when they are apart they are at a higher potential energy then when they are together.

Here's an analogy:
Physically, this situation is like having a ball at the top of a hill overlooking a valley or well. The ball will roll down the hill and that potential energy is converted into kinetic energy. When the ball reaches the bottom of the valley it will start climbing back out of the well and turn that kinetic energy back into potential, so if the ball starts at rest it only gets back to being as high as it started. However in the real world there is friction that will steal some of this kinetic energy and so the ball will roll back and forth, but eventually come to rest at the bottom of the hill.

For the electron an proton you'll see something similar. The 2 particles will accelerate towards each other, pass/scatter off each other (and then repeat) and will slowly lose energy to "friction" i.e. to radiation.

Part 2: Specific questions

1) Do they collide and bounce off? (conserving momentum) 2) Does the electron get through the proton, i.e. between its quarks?

The collision between the two particles is perfectly elastic. In addition the energies (~13eV) are so small relative to the strong force holding together the proton that quarks are not involved in any way, and the scattering is described by Rutherford scattering.

3) Do both charges give off Brehmsstrahlung radiation while moving towards each other?

The 2 particles will radiate and lose their kinetic energies. The term Brehmsstrahlung is generally reserved for much higher particle energies (>keV), and much larger accelerations.

Suppose I can control the two particles, and I bring them very close to each other (but they are not moving so quickly as before, so they have almost no momentum). Then I let them go:

1) Would an atom be spontaneously formed?

You can immediately describe the 2 particles by their center of mass description (an atom) plus their individual attributes (i.e. what the particles are doing within the atom). Assuming the 2 particles start off at rest, then they are in a bound state already because they can't escape each other (go off to infinite separation) due to lack of energy.

However the atom will not be in it's ground state until it has decayed into the lowest level via spontaneous emission of radiation.

2) If anything else happens: what kind of assumptions do we make before solving the TISE for an Hydrogen atom? Does the fact that the electron is bound enter in it? This is to say: is quantum mechanics (thus solving the Schrödinger equation) the answer to all my questions here?

The TISE of the atom itself will give you energy levels etc, but you will not get spontaneous emission into the ground state unless you put it in by hand (and it would not be time independent anymore) or also quantize the EM vacuum (which is how you derive SE). So trying to solve it would be like solving the ball moving on the hill while ignoring friction, it will just oscillate at constant energy forever.