Int Vs. Char For Datatype On Numeric Label?

I'm designing a database that will not be that large (I would be surprised if it ever surpassed 50,000 rows across all tables), but will be accessed quite often, so I am doing my best to optimize its structure, such as doing 3NF, selecting appropriate data types, etc.

I have a few columns that will contain numeric data (such as an invoice number (from an external source) or location ID). However, one of my classes in college was about database design, and we were taught that if you won't be doing mathmatic computation on a field (such as the invoice or location fields I mentioned earlier), then you should use a string literal type (char, varchar, etc.)

Unfortunatly, the professor did not explain much as to why this should be done. From the standpoint of semantic analysis, these types of fields are probably more labels than they are numbers. However, I don't find that very convincing (or even helpful).

In short, my question is that given a column holding numeric data that isn't worked on in a mathematical sense, is it really better to mark it as a string literal than a number? Any articles or studies I can read relating to this?

I would think that comparisons would be faster with int, as well as data storage (though, as mentioned, that's not as big of a concern). Sadly, Google doesn't have many helpful resources. (Lots of stuff on char vs varchar, though.)