Fix column calculation when a comment does not end in a newline.

Previously the column update always assumed a newline immediately
followed the end of a comment. In fixing the column calculation the
comment handling code in the tokenizer has been simplified and
consolidated.