Each line you enter will be parsed into tokens. > :a Op(:) Id(a) EndMarker() > -a Op(-) Id(a) EndMarker() > >a Op(>) Id(a) EndMarker() > foo Id(foo) EndMarker() > foo_bar Id(foo_bar) EndMarker() > bar_ Id(bar_) EndMarker() > 1.2 Real(1.2) EndMarker() > 2E10 Real(2e10) EndMarker() > 2e5 Real(200000) EndMarker() > 3E-9 Real(3e-9) EndMarker() > 3.4E5 Real(340000) EndMarker() > 5.6e-7 Real(5.6e-7) EndMarker() > 1. Real(1) EndMarker() > 2.e2 Real(200) EndMarker() > .123 Real(0.123) EndMarker() > .123e2 Real(12.3) EndMarker() > 0000.123 Real(0.123) EndMarker() > (* comment *) Comment((* comment *)) EndMarker() > (* (* nested *) comment *) Comment((* (* nested *) comment *)) EndMarker() > "Text Literal" Text("Text Literal") EndMarker() > "Hello, World!\n" Text("Hello, World!\n") EndMarker() > "Escape characters: \n\f\t\r\"\000\001\002" Text("Escape characters: \n\f\t\r\"\000\001\002") EndMarker() > ^D