As you have likely heard drone on repeatedly in this space, I was a failed congressional candidate back in 2008, when I pulled in more than 113,000 votes, which might seem like a lot until you know that the GOPer involved, U.S. Rep. Doug Lamborn, got 183,000 or so. Still, about as many people voted for me as cram into the University of Michigan stadium when they play Ohio State (Go Blue!)

When I first declared my interest in running for office, I was advised I should start out running for a smaller office, perhaps the state legislature. I wasn’t interested in that, though in hindsight, a recent Colorado Politics article suggests I might have had more success pursuing a win at that lower level. If I played my cards right, I might win a seat in the state House by getting only four votes.

Let me explain that crazy number. As with any organization, including all the various state legislatures around the nation, vacancies occur from time to time. Sometimes we see members die in office, and that’s sad. Sometimes they resign their seats to pursue other opportunities, often to run for seemingly higher-ranking electoral offices. As a result, in most state legislatures a certain number of the members are folks appointed to fill a vacancy, rather than people who have faced the voters.

Stay up to speed: Sign up for daily opinion in your inbox Monday-Friday

In Colorado we have 100 members in our state legislature, with 65 in the state House and 35 in the state Senate. According to a recent CoPo analysis, of that 100, amazingly 28 are, or were, appointed to office. Some have since stood in a subsequent election, but roughly a third of our legislature came to office by appointment, and not by the will of the people, filling a seat that had been vacated for some reason.

And what is rather shocking is how few people it takes to “win” an appointment to fill a vacancy. In many cases, the district vacancy committee of the political party involved can be just a few dozen, perhaps 40 folks. And in some extreme cases, the number involved in picking who gets to fill a vacant seat can be as low as, yup, four people. Republican state Sen. Perry Will needed two-thirds of the vacancy committee’s votes to be selected as a replacement representative but given that the replacement committee was only six people, the magic number was, well, as a I said, four. Surely no one can argue such a process is fundamentally democratic.

Different states have different rules on how to handle legislative vacancies. Only five, Colorado, Illinois, Indiana, New Jersey and North Dakota (or as a friend of mine who lives there calls it, “Baja Manitoba”), handle it like the Centennial State. The other 45 states use a variety of methods. Eleven have direct appointment by the governor, which is great if you are of the same party as the governor but is less attractive if that isn’t the case. Eight others have county commissions pick, and the remaining 25 have special elections.

And that’s what we should have: special elections.

For Colorado, the current system means nearly a third of our representatives were picked in the metaphoric smoke-filled rooms of old. They aren’t smoke-filled anymore, but it is still a small group of party activists who are imposing their will over the will of the roughly 89,000 people in a state House district or the 165,000 folks in a state Senate district. That hardly seems fair.

Elected officials step down for a variety of reasons. Some in the House resign to be appointed to a vacant Senate seat. Others leave office because of opportunities elsewhere, sometimes in government and sometimes in the private sector. Some, sadly, die or become ill, and a couple recently have stepped down out of disgust with the vitriol of the current session. But for whatever reason a person steps down, it is nuts a relatively small group, ranging from a central committee of perhaps a hundred, to the aforementioned gang of six, gets to pick the representative of an entire district.

So why not special elections? Well, the first objection is always going to be cost. Though our mail-in ballot system is cheaper than “regular” in person elections, it is still not free. Additionally, others will point out with our rather short legislative session here in Colorado of around 120 days, there isn’t any reason to not just appoint a person to get through the session, with many months after for that appointee to decide if he or she wants to run in the next general election.

And I get that. But if free and fair elections are not worth spending taxpayer dollars on, what in the name of Alexander Hamilton is? The notion people should select their leaders through elections is the quintessential American belief. Recall please through the entire history of humanity, up until Thomas Jefferson put quill to paper in 1776 and drafted the Declaration of Independence, the leaders of nations declared themselves to have been picked by God. To challenge a king or a pharaoh, or a shah, or any other such leader was to challenge God. In declaring our independence from England, we also declared we would pick our own leaders, the king be damned.

Elections are not always perfect, and they are not without cost, but they are the core of what it means to truly be an American. To pick your own leaders is to stay free. And such decisions should not come down to a half-dozen folks picking our representatives.

So yes, we should change our system of filling vacancies and we should spend the money necessary to hold free and fair elections to pick our leaders.

It is, fundamentally, the American thing to do.

Hal Bidlack is a retired professor of political science and a retired Air Force lieutenant colonel who taught more than 17 years at the U.S. Air Force Academy in Colorado Springs.