Why is the periodic table organized by atomic number rather than alphabetically?

It's because with similar atomic numbers have similar properties.

You know that there are several types of elements. Non-metals, noble gases, alkaline metals, etc. They are formed into groups because of their behavior. The plays a part in this. Elements that have similar atomic numbers are in groups. The lanthanides aren't scattered across the table, they're very close to one another in a group. For example, the noble gases all have full outer shells, making them exhibit similar characteristics such as not reacting.

The atomic number also represents the number of protons (and electrons) in each element, and these protons and electrons dictate how the elements act, in terms of things like bonding, size, and many other characteristics.

Sorting alphabetically, however, would be unhelpful and arbitrary since elements can be named anything. Using atomic number actually represents and sorts the atoms by their most basic characteristic.