We show that the André motive of a hyper-Kähler variety X over a field K⊂ C with b2(X) > 6 is governed by its component in degree 2. More precisely, we prove that if X1 and X2 are deformation equivalent hyper-Kähler varieties with b2(Xi) > 6 and if there exists a Hodge isometry f: H2(X1, Q) → H2(X2, Q) , then the André motives of X1 and X2 are isomorphic after a finite extension of K, up to an additional technical assumption in presence of non-trivial odd cohomology. As a consequence, the Galois representations on the étale cohomology of X1 and X2 are isomorphic as well. We prove a similar result for varieties over a finite field which can be lifted to hyper-Kähler varieties for which the Mumford–Tate conjecture is true.