Quantcast

Re: Issues with Unicode and file import

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Issues with Unicode and file import

Michael Chinen
On Mon, May 20, 2013 at 7:09 PM, Gale Andrews <[hidden email]> wrote:

>
> | From David Ryskalczyk <[hidden email]>
> | Sun, 19 May 2013 00:09:11 -0400
> | Subject: Issues with Unicode and file import
>> After further testing I found that it's very intermittent even with
>> that file, but it happens often enough to cause issues. I wish this
>> was more reproducible.
>>
>> The crash report does match mine, though, so maybe that would help.
>> I'd be guessing memory corruption, but that's just a wild guess — I
>> haven't looked at any of the code yet. I can send my crash reports
>> tomorrow.
>
> Hi David,
>
> I expect you've seen Michael tried a fix for this:
> http://code.google.com/p/audacity/source/detail?r=12334
>
> which I've tried out on my Mac. If you don't regularly build
> Audacity yourself, you can get the "nightly" containing that fix
> from here:
> http://www.audacity.homerow.net/index.php?dir=mac%2Faudacity-nightly-2013.05.20-03.15%2F .
>
> Summary: I see a repeatable problem with adding spaces into
> the name of the suspect file, then using Open with.
>
> Detail: I did not get it to fail until I worked with the copy of your
> file where I had already right-clicked over it in Finder > Get Info,
> copied the "Name" then paste-overwritten the file name.
>
> As it stood that file opened fine with Michael's patch, no matter
> what method I used to open the file.
>
> (A) So then I selected the file in Finder, ENTER (which opens the
> file name for editing), then pressed SPACE on the keyboard to
> add a space between the first two oriental characters. With
> Audacity quit, I right-clicked over the just renamed file > Open
> with, and chose the patched build. I saw the Welcome Screen,
> but no imported file, then Audacity crashed with a similar looking
> error trace to before (see inline).
>
> (B) I reopened the build without opening a file, then imported an
> MP3 without Unicode characters in the path, then right-click over
> the renamed file > Open with. The crash occurred again with the
> same error trace.
>
> I repeated procedure (B) six times (quitting and launching
> Audacity afresh each of the six times) with no further crash
>
> Now I repeated (A) by adding a space between second and third
> characters in the file name. I got the same crash.
>
> Now I repeated (B) twice with crashes, then repeated (B) again
> with no crashes. As soon as I repeated (A) by adding a space
> between other characters, crash.
>
> I rebooted and I get the same problem - adding spaces in the
> file name causes a crash when using right-click > Open with.
> It usually also crashes after rename by dragging in, but not
> always.

Further investigation shows this bug exists with or without the space.
 It happens about 1%-5% of the time for me now.  The way I trigger it
is to create a short wav file with the same katakana name and make 20
copies of it, and drag those all at once into an audacity window until
the bug triggers.
I tested some more and I was able to trigger it without a unicode
filename, after importing unicode filenames previously, with one as
simple as "aaa" when importing enough (~50 tries).

I spent some time looking into possible causes and have not come up
with anything.  It appears occasionally the wxLogMessage call to
VPrintf does the wrong thing converting the asterix mime-type wxString
and creates a bad string that isn't terminated.  For me, this always
happened at the same line - on the mime type log message after the
file name log message line.

I tried a few things, but what seemed to make things better was
changing all c_str() to mb_str() for Import.cpp/Project.cpp, as in,
this actually had an effect on where the crash would occur - it seemed
to be a format string function after a previous c_str() call on a
unicode string.  but I couldn't replace c_str() globally because there
are too many cases in too many files, but this could be a potential
fix.

For now I just thought I'd post my findings.

Michael

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
audacity-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/audacity-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Issues with Unicode and file import

Gale
Administrator
"Michael Chinen" wrote:
>> On Mon, May 20, 2013 at 7:09 PM, Gale Andrews <[hidden email]> wrote:
>>
>> Summary: I see a repeatable problem with adding spaces into
>> the name of the suspect file, then using Open with.
> Further investigation shows this bug exists with or without the space.
> It happens about 1%-5% of the time for me now.  The way I trigger it
> is to create a short wav file with the same katakana name and make 20
> copies of it, and drag those all at once into an audacity window until
> the bug triggers.

Thanks, Michael for looking further at it. As I just said in the "Bug Hunt
Sunday" thread, I'm down to only about 50% reproducible now. I only
just noticed this thread otherwise I'd have only replied here.    


> I tested some more and I was able to trigger it without a unicode
> filename, after importing unicode filenames previously, with one as
> simple as "aaa" when importing enough (~50 tries).
Were there any Unicode characters anywhere in the path? Files with
Latin character names in folders with Unicode names are also a reported
problem.  

Is there any relevance to the act of changing the name, then
reimporting the contents? That was replicating so reliably that it
looked more than coincidence. I have just got a crash by changing
a file called "bbb" to "bbbz" in a completely Latin path and then
right-click > Open with (after successfully importing a file with a
Unicode name). However the crash report looks identical to the
report after importing a file with a Unicode name.  


> I tried a few things, but what seemed to make things better was
> changing all c_str() to mb_str() for Import.cpp/Project.cpp, as in,
> this actually had an effect on where the crash would occur - it seemed
> to be a format string function after a previous c_str() call on a
> unicode string.
OK I'll try that on my machine if not advised otherwise.  


> I couldn't replace c_str() globally because there are too many cases in
>  too many files, but this could be a potential fix.
If I'm reading this correctly:
http://docs.wxwidgets.org/trunk/overview_changes_since28.html#overview_changes_unicode 

we may have to change a lot of c_str() instances when we upgrade to
Widgets 3.x.  


Gale

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Issues with Unicode and file import

Michael Chinen
On Wed, May 22, 2013 at 10:02 AM, Gale (Audacity Team)
<[hidden email]> wrote:

> "Michael Chinen" wrote:
>>> On Mon, May 20, 2013 at 7:09 PM, Gale Andrews &lt;gale@&gt; wrote:
>>>
>>> Summary: I see a repeatable problem with adding spaces into
>>> the name of the suspect file, then using Open with.
>> Further investigation shows this bug exists with or without the space.
>> It happens about 1%-5% of the time for me now.  The way I trigger it
>> is to create a short wav file with the same katakana name and make 20
>> copies of it, and drag those all at once into an audacity window until
>> the bug triggers.
>
> Thanks, Michael for looking further at it. As I just said in the "Bug Hunt
> Sunday" thread, I'm down to only about 50% reproducible now. I only
> just noticed this thread otherwise I'd have only replied here.
>
>
>> I tested some more and I was able to trigger it without a unicode
>> filename, after importing unicode filenames previously, with one as
>> simple as "aaa" when importing enough (~50 tries).
> Were there any Unicode characters anywhere in the path? Files with
> Latin character names in folders with Unicode names are also a reported
> problem.
>
> Is there any relevance to the act of changing the name, then
> reimporting the contents? That was replicating so reliably that it
> looked more than coincidence. I have just got a crash by changing
> a file called "bbb" to "bbbz" in a completely Latin path and then
> right-click > Open with (after successfully importing a file with a
> Unicode name). However the crash report looks identical to the
> report after importing a file with a Unicode name.
>
>
>> I tried a few things, but what seemed to make things better was
>> changing all c_str() to mb_str() for Import.cpp/Project.cpp, as in,
>> this actually had an effect on where the crash would occur - it seemed
>> to be a format string function after a previous c_str() call on a
>> unicode string.
> OK I'll try that on my machine if not advised otherwise.

I looked a bit more into this and found there are conflicting
documentation from wxWidgets about which one to use.  Some
documentation suggests c_str() is the correct choice.  To test, I
tried converting more to either both mb_str() and c_str() using lots
of regexes but it seems pretty complicated to get correct, and in some
cases it seems like they require one or the other.  In my tests, they
both produce crashes in string conversion, but in different places.

I found some ML posts about how unicode string conversion crashes with
wxMac was resolved by using later mac OS SDK versions like 10.6, but I
haven't gotten around to trying it out yet.
Or, it could be related to this crash that was fixed in the 2.9 branch
http://trac.wxwidgets.org/ticket/13442

Not sure when I will look at this again, but I just wanted to post in
case someone else does first.

Michael

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
_______________________________________________
audacity-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/audacity-devel
Loading...