Re: [gentoo-user] utf8_general_ci

[Fernando Rodriguez]

On Tuesday, May 05, 2015 11:03:38 AM Joseph wrote:
> On 05/05/15 12:32, Fernando Rodriguez wrote:
> >On Tuesday, May 05, 2015 9:32:15 AM Joseph wrote:
> >> I have my mysql database "Collation" set as: utf8_general_ci
> >>
> >> but when a customer from for example Japan places an order all I see is:
> >>
> >>
>
>竹鼻立原町５－５
> >>
> >> Do I need to change "Collation" setting to something else or something
else?
> >>
> >>
> >
> >I think that's because the web applications runs the data through something
> >like php's htmlspecialchars() or similar to help prevent SQL injections. So
> >you'll need to either decode it before using it (I think you can use the
app-
> >text/recode), or use a different method to filter anything that could be
> >malicious SQL.
>
> I've saved the relevant information into a TXT file (address.txt) and tried
to run: recode ISO-8859-9..UTF8 < address.txt > address2.txt
>
>
&amp;#31481;&amp;#40763;&amp;#31435;&amp;#21407;&amp;#30010;&amp;#65301;&amp;#65293;&amp;#65301;
> &amp;#23665;&amp;#31185;&amp;#21306;
> &amp;#20140;&amp;#37117;&amp;#24066;, 601-8015
> &amp;#20140;&amp;#37117;&amp;#24220;, Japan
>
> It didn't help.  How do you run "recode" correctly?
> Yes, the customer is using oscommerce php addlication to provide
information.

It looks like they ran it through the encoding function twice. This worked for 
me:

recode html..utf8 < test.txt | recode html..utf8


-- 
Fernando Rodriguez