I have to emphasize that I am not a scholar on Hilbert, mathematics, or philosophy, so my explanations are not necessarily always perfectly on target, and at a certain depth, I would have to defer to people who have studied more extensively than I have. And I don't mean necessarily to defend Hilbert's philosophical notions in all its aspects.
That said, however, here's a stab at answering your question:
I think what Hilbert has in mind is the distinction between a) reasoning with symbols that are taken as representing particular numbers and b) making generalizations about an infinite class of numbers.
For example, if 'a' is a token for a particular number, then the truth of 'a+1 = 1+a' cannot be reasonably contested as it can be concretely verified - it is finitistic. For example, for the particular numeral '2', the truth of '2+1 = 1+2' cannot be reasonably contested as it can be concretely verified.
On the other hand, where 'A' stands for any undetermined member of entire infinite class of numbers, then 'A+1 = 1+A' (which is ordinarily understood as 'for all numbers A, we have A+1 = 1+A') cannot be verified concretely because it speaks of an entire infinite class that we can't exhaustively check. Therefore, some other regard must be given the formula. And that regard is to take it as not "contentual" but as "ideal" but formally provable from formal axioms (which are themselves "ideal"). And it is needed that there is an algorithm that can check for any purported formal proof that it actually is a formal proof (i.e., that its syntax is correct and that every formula does syntactically "lock" in sequence in applications of the formal rules); this is what Hilbert has in mind as the formal "game". Then Hilbert hoped that there would be found a formal proof, by using only finitistic means, that the "ideal" axioms sufficient for ordinary mathematics are consistent. Godel, though, proved that Hilbert's hope cannot be realized.