Japanese Unicode

[humasect]

Using Japanese characters (either katakana or hiragana) in identifiers rules this:

Source/Hehe.hs:12:0: lexical error at character '\12390'

There is no issue with Haskell98 for upper/lower case identifiers and type constructor identification with the two complimenting Japanese character sets. Using -fglasgow-exts along with other Unicode characters for various operators which work great.

[simonmar]

Please attach some example code illustrating the bug.

BTW, the "priority" field of the ticket is mainly for the GHC developers so we can prioritise tickets; please use "severity" to indicate how badly the bug affects you. Someday I'll figure out how to put a link to some docs next to these fields on the ticket page.

[humasect]

1 working out of 3 unicode tests

[humasect]

My apologies. I've attached some test code. We could really use Japanese identifiers in house development. I don't know what to say about upper/lower case for identifiers and constructors. I could create some sort of example code for conventions that would work very well. Thanks again

[ross]

I don't think there's any reason why these characters couldn't be treated as upper or lower case letters; the question is which. We'd want to treat all the members of a Unicode General Category the same way, because special cases would be too cumbersome. A lexical syntax based on the case of letters was never going to work well with caseless scripts.

Kanji, katakana and hiragana all belong to the Letter, Other category. If we treated these as lower case, your third example would work, but you'd have to adopt a convention of prepending capital letters (like M, C, T and D) to Japanese module, class, type and data constructor names. (The same would apply to all the other caseless scripts too.)

[igloo]

Another option would be to treat them as neither upper nor lower case, so they could be part of a name but not the first character of it. I think treating them as lower case make more sense, though. Whatever we do, we should make sure Haskell' matches it.

[simonmar]

Punt to 6.8: this requires further thought and coordination with Haskell'.

[humasect]

Her is an idea/proposal for some kind of simple extension to also allow backward-compatible "international" source code. Multilingual language ?

[igloo]

Punt to 6.10 as this still requires further thought and coordination with Haskell'.

[simonmar]

I did as Ross suggested and made the "Letter, Other" class behave as lower-case.

Wed Jul  9 10:12:52 BST 2008  Simon Marlow <marlowsd@gmail.com>
  * Treat the Unicode "Letter, Other" class as lowercase letters (#1103)
  This is an arbitrary choice, but it's strictly more useful than the
  current situation, where these characters cannot be used in
  identifiers at all.
  
  In Haskell' we may revisit this decision (it's on my list of things to
  discuss), but for now this is an improvement for those using caseless
  languages.