Gowers (op cit) has a brief section on "infinity" in which he shows there is no number system satisfying the ordinary axioms that also has a number $\infty $ such that $\infty \times 0=1$. The argument is,

Thus it requires only associativity of $\times $, and a distinct number $2$ with $0\times 2=0$ and $1\times 2=2$.

Here, of course, $\infty $ is considered as a *number* in the system.

Gowers also has a whole chapter entitled
"Limits and Infinity". He says, "the concept of infinity
is indispensible to mathematics, and yet is a very hard idea to make rigorous."
The chapter discusses the possibility of a rigorous
picture of infinity via the concept of *limit*.

So here, "infinity" is considered as a *process* rather than a number.

The distinction (or lack of distinction when one should be made) between infinity as a number and infinity as a process causes a great deal of confusion, aggravated by sloppy language such as "and so on, going to infinity", as if infinity is somewhere one can go, or the (sometimes justifiable) use of the symbol $\infty $ in calculations, for example,

What actually is a process in mathematics? To be a process
requires some concept of *time*, and yet mathematics when done
rigorously has no concept of time, unless time is a special variable
letter such as $t$, and then the mathematics is done for a fixed
(but general) value of this $t$.

Mathematicians do have an informal conventional language that involves time in some informal sense: we might say, "let $x=x\left(t\right)$, so $x$ varies with time $t$," but $t$ and hence $x$ are fixed in what follows. Or "consider the sequence ${a}_{n}=1/{2}^{n}$ as $n$ increases" suggesting the $n$ is the measure of "time".

A sequence is of course just a notation for a function
$\mathbb{N}\to \mathbb{R}$. Thus processes are typically represented as
*functions* such as $f\left(t\right)$ or ${a}_{n}$ in
mathematics and we study processes via the study of functions at
specific and fixed but general values for their arguments $t$ or
$n$.

When we study functions in this way we typically want to abstract some property of these functions, such as their behaviour as the argument "approaches infinity". It seems to be natural to introduce a symbol $\infty $ for this "object" that the argument "approaches", and then natural to look at the limiting value notated $f\left(\infty \right)$ or ${a}_{\infty}$.

Unfortunately this does not make literal sense ($\infty $ is not a possible argument for the function) nor could it make mathematical sense. For example, if ${a}_{n}={\left(-1\right)}^{n}$ is ${a}_{\infty}$ equal to $\mathrm{+1}$ or $-1$ (or something else?).

The official solution is to say that the intuition is wrong and
no number $\infty $ could behave like this. In its stead we must carefully
analyse the concept of *limit* of a sequence. The definition is
that a sequence ${a}_{n}$ has a limit if

What a pity we can't just consider a number $\infty $ and say that ${a}_{n}$ has a limit if ${a}_{\infty}$ is well defined.

That is the mainstream view. But there is a non-standard nonstandard
view that you can do exactly this. This is Abraham Robinson's
"nonstandard analysis" and is "nonstandard" in the technical sense
that it is about such "nonstandard" numbers but it also non-standard in
the literal sense that most mathematicians regard it as unnecessary, not standard,
and a highly unusual discipline to follow, albeit one that is perfectly rigorous and
acceptable in the sense of that word. I'd even go so far that most mathematicians
(who typically are rather platonistic about their belief in numbers) would not
even claim such nonstandard numbers to have (platonic) existence *as
numbers*, though they do admit them as a possible fiction which (when
necessary) can be seen as names for certain technical mathematical
constructions involving ultrafilters. (Don't worry: I will explain what these are.)