The two best answers to "what is a startup"?

Steve Blank - What’s A Startup? First Principles. [2010]

Your startup is essentially an organization built to search for a repeatable and scalable business model. As a founder you start out with:

1) a vision of a product with a set of features,

2) a series of hypotheses about all the pieces of the business model: Who are the customers/users? What’s the distribution channel. How do we price and position the product? How do we create end user demand? Who are our partners? Where/how do we build the product? How do we finance the company, etc.

Your job as a founder is to quickly validate whether the model is correct by seeing if customers behave as your model predicts. Most of the time the darn customers don’t behave as you predicted. 

Paul Graham - Startup = Growth [2012]

A startup is a company designed to grow fast. Being newly founded does not in itself make a company a startup. Nor is it necessary for a startup to work on technology, or take venture funding, or have some sort of "exit." The only essential thing is growth. 

To grow rapidly, you need to make something you can sell to a big market. That's the difference between Google and a barbershop. A barbershop doesn't scale.