Time is one of the words in English that can be either countable or uncountable.
There is not enough time to finish.
He spends most of his time with her.
At the time of their arrest, they were drunk.
He kicked the ball several times
In colonial times, there were few printing presses.
She went to the movie four times.
These times are difficult.
What seems to be going on here is that when time refers to an abstract idea, it is uncountable. It is a general idea that can be broken down into countable units like hours, or minutes. Thus, this use of time is like money. Time is a large or abstract category.
But when times refers to a concrete action or period of time, it is countable. In this case, it is not broken down into smaller units.
He saw her several times last week.
Here it is countable because the times were separate units (each time he saw her) and countable. Unlike spending most of his time, in which case, the time could be broken down into smaller units such as hours or minutes.