Classification of $O(2)$-bundles in terms of characteristic classes
The $O(2)$ bundles $\xi$ over a manifold $M$ are classified by their first Stiefel-Whitney class $w_1(\xi)\in H^1(M;\mathbb{Z}/2)$ and their twisted Euler class $e(\xi)\in H^2(M;\mathbb{Z}_{w_1(\xi)})$.
This is because the space $BO(2)$ is a generalized Eilenberg--Mac Lane space $L_{w_1}(\mathbb{Z},2)$ in the sense of
Samuel Gitler, Cohomology operations with local coefficients, Amer. J. Math. 85 (1963), 156--188.
In (slightly) more detail, there is a fibration $$ K(\mathbb{Z},2)\to E\mathbb{Z}/2\times_{\mathbb{Z}/2} K(\mathbb{Z},2)\to B\mathbb{Z}/2 $$ given by the twisting of $w_1$ on the universal $SO(2)$ bundle, and this fibration agrees up to homotopy with the fibration $$BSO(2)\to BO(2)\to BO(1).$$
To complement Mark Grant's excellent answer, I'll say something more about the general case. This topic goes under the name of obstruction theory.
The first observation is that a $G$-bundle on $X$ is the same thing as an homotopy class of maps $X\to BG$. To study them we will use the Postnikov tower of $BG$. This is a tower assembled by spaces $P_n(BG)$ together with a map $BG\to P_n(BG)$ such that
- The map $\pi_i(BG)\to \pi_i(P_n(BG))$ is an isomorphism for $i\le n$;
- $\pi_i(P_n(BG))=0$ for $i>n$.
We can assemble this spaces together so to form a tower as follows:
$\require{AMScd}$ \begin{CD} @. \vdots\\ @. @VVV \\ @. P_2(BG)\\ @. @VVV \\ X @>>d> P_1(BG) \end{CD}
and moreover the limit of the tower is $BG$. So we can study the homotopy classes $[X,BG]$ by studying the collections of arrows $[X,P_i(BG)]$ making the diagram commute.
Now let's start at the bottom of the diagram. By definition we have that $P_1(BG)$ is a $K(\pi_1BG,1)=K(\pi_0G,1)$, so we have
$[X,P_1(BG)] = [X,K(\pi_0G,1)] = H^1(X;\pi_0G)$
This is our first cohomology class, corresponding to $w_1$ in the case of $BO(n)$.
Now let us suppose that we have lifted our map all the way to $P_n(BG)$ and we want to see what algebraic information corresponds to a lift to $P_{n+1}(BG)$. It turns out that there is a cartesian diagram $\require{AMScd}$ \begin{CD} @. P_{n+1}(BG) @>>> K(\pi_0G,1)\\ @. @VVV @VVV\\ X @>>> P_n(BG) @>>> K(\pi_{n+1}G,n+2)_{h\pi_0G} \end{CD} (don't be scared by all those homotopy quotients you see: they're just the homotopy theorist's way of saying that we're dealing with twisted cohomology classes). So lifting a map from $P_n(BG)$ to $P_{n+1}(BG)$ is the same thing as lifting a map from $K(\pi_{n+1}G,n+2)_{h\pi_0G}$ to $K(\pi_0G,1)$. This is saying that the lift exists if and only if some class in $H^{n+2}(X,\pi_{n+1}G)$ vanishes (not all choices of characteristic classes will correspond to a $G$-bundle!) but, more importantly for us, this is exactly the same situation as in Mark Grant's answer and so the possible choices are parametrized by a class in $H^{n+3}(X,\pi_{n+1}G)$.
So, to sum up we will have
- A class $\alpha$ in $H^1(X;\pi_0G)$
- An infinite sequence of classes in $H^{n+1}(X;\pi_nG)$ for $n\ge1$ where the coefficients are twisted by $\alpha$.