[funsec] Things You Can't Say in Notepad

[Nick FitzGerald]

Rob, grandpa of Ryan, Trevor, Devon & Hannah wrote:

> > Not sure if this helps any investigations, but we've confirmed that it
> > _does not_ work under Wine.  I guess it must be an API problem.
> 
> Molto bizarre, if what Nick says about Unicode detection is on the mark.

It _is_ Unicode detection weirdness.  Not sure if this URL has been 
posted here yet, but MS' "Technical Lead" for localization, NLS, etc, 
Michael Kaplan covers it in his blog:

   http://blogs.msdn.com/michkap/archive/2006/06/14/631016.aspx

Basically it is the due to Notepad's use of the perrenially borked 
IsTextUnicode API to test if a file is Unicode (and presumably 
Notepad's use of test flags for "guessing" this independent of starting 
with Unicode byte order marks).

The "recently" and "in the past" links in that blog entry are both 
worth reading if you are interested in the technical history (and 
dodginess) of all this...

This presumably also explains why it works in Notepad2 (a quick grep of 
its source shows calls to IsTextUnicode but I'm not going to try to 
decode the dox and flags), and Wine presumably does not implement the 
fully borked IsTextUnicode (maybe its code only checks for BOMs and/or 
only uses other "strong" tests, rather than supporting all the fast-and-
loose checks of the real thing?).


Regards,

Nick FitzGerald