How does the CSGO penalty system work?

The ban systems works as follows:

1st infraction: 30 min.
2nd infraction: 2 hours.
3rd infraction: 24 hours
4th infraction: 7 days

Any one of these infractions get reset after 7 days.

How the banning system works (To the extent of my knowledge):

1. Dealing 80/90+ damage to a teammate at start of game will be instant ban.
2. Doing 400+ of damage to teammate(s) over the period of the whole game can get you banned. (You usually get a warning telling you that you have done a lot of damage and the next time you attack a friendly you will be kicked)
3. Killing 3 teammates

How you got kicked and he didn't: He didn't get kicked purely because it wasn't at the start of the game and in spawn. When it's like that the game knows that there are enemies around and sometimes that you are likely to either accidentally attack your team mate or maybe even kill him. For this they make exceptions. However if you were to do it as a continuous burst of fire (so all his bullets hit you without any pause) then he would have gotten kicked/banned.

In cases like that all can really do is report him through the use of the scoreboard as someone has mentioned in the comments section.

Competitive matchmaking cooldown offences:

• Abandoning: leaving the match at any point before the end (warm-up included)
• Idling / AFKing: after a set period of time (not sure how long) with no activity you will be removed
• Excessive team-kicking: unsure of how many kicks constitutes excessive, but this does not result in a removal, only a cooldown
• Excessive team-killing: after team killing for the third time in one match you will be removed
• Excessive team-damaging: after dealing a set amount of damage in one match you will be removed (believed to be around 500 and includes over-kill damage)
• Excessive team-damaging at the start of a round: the damage thresh-hold is reduced at the start of the round. You can therefore be kicked for doing less than normal damage thresh-hold in the first moments of the round (unfortunately I don't know how long this lasts for or what it is reduced to)

Regarding excessive team-damage and over-kill. You will often see a player team kill with an AWP to the head. This results in about 400 damage and means the next team damage caused by that player normally results in a kick.

Cooldown offence levels and lengths:

• Level 1: 30 minutes
• Level 2: 2 hours
• Level 3: 24 hours
• Level 4: 7 days

Cooldown resets:

• Every cooldown received increases your current offence level by 1 (your offence level starts from 0)
• Offence levels can be decreased by 1 by going 7 days without receiving a cooldown
• The 7 days does not begin until your current cooldown has expired
• Offence levels can rise above level 4, but 7 days is the maximum cooldown duration
• Confusion can arise for extreme abusers when the offence level goes over 4

An example:

A player reaches offence level 4 and gets their first 7 day cooldown on January 1st. The cooldown expires on January 8th. The player can now play in competitive matchmaking again, but their offence level is still 4.

They receive another cooldown on January 9th and receive another 7 day cooldown (because they were still offence level 4), their offence level rises to 5. Their ban expires on January 16th and they don't commit any offences for a week. Their offence level drops back to 4.

Essentially, if you get your second or third 7 day cooldown in a row, it means you need to have two or three clean weeks after it expires before your offence level drops below 4 and therefore will receive cooldown lengths other than 7 days.

Source:
Cooldown reset information