There are many ways to build a computer, so why do we use silicon? It all boils down to electrochemistry, the same rules that govern the neurons in our brains. Ultimately, whatever material we use, needs to be easy enough for humans to control - and binary enough to send out information in its most practical form, the 1 or 0.
Not enough control. Quantum computing.
For a long time, the smallest individual unit of matter was an atom and the smallest unit of information was a bit. That’s been challenged in the last century because once you get smaller than atoms, into the subatomic realm, the world begins to run on quantum mechanics, which exhibits “quantum weirdness” - when our intuition about the way the world works no longer applies at the quantum scale.
No longer are things binary, but rather information exists in qubits (“quantum bits”) which are the quantum spin states of various types of subatomic particles. In fact, in the quantum world, the mere act of knowing something negates your knowing it. Things no longer exist in binary, they exist in “both 1 and 0 at the same time” or “kind of 1, almost 0” - in other words, it’s weird. And it’s so small that we can’t really observe or control it. Yet.
Too much effort. Magnetic computing.
Magnetism is another way of computing that is very binary, but requires a huge amount of power to set the initial state. According to the rules of electromagnetism, where electricity generates a magnetic field and vice versa - you have to be really careful about having both things too close to each other. There are magnetic chips made of tantalum 
that have very low power requirements, but because they are hard to manufacture at scale, they pale in comparison to existing silicon transistor chips.
So now we get to the answer, why silicon? And what is a transistor?