After installing Jenkins from the repositories given on the site, I attempted to start the service by running:

$ sudo service jenkins start

The problem is that even though it spits out the error message:

The selected http port (8080) seems to be in use by another program
Please select another port to use for jenkins

It ends by telling the user that everything went just as expected:

• Starting Jenkins Continuous Integration Server jenkins [ OK ]