Neighbourhoods
There are a number of different neighbourhoods you might use, of different radii.
The simplest neighbourhood is the Moore Neighbourhood:
The other common neighbourhood is the Von Neumann Neighbourhood:
Rather less usual is the Diagonal Neighbourhood:
Of course, you can also have a wide variety of other regular and irregular shapes, but these are the most common.
The other aspect that changes with neighbourhoods is whether the central cell is examined or not. This depends on the algorithm it is being used with. For example, with Conservative Filters the central cell is excluded, whereas for the Mean Filter it is usually included in the calculation of the neighbourhood mean.