Changeset 64b86be in osmose-backend


Ignore:
Timestamp:
Oct 9, 2014 2:21:21 AM (5 years ago)
Author:
Nelson A. de Oliveira <naoliv@…>
Branches:
master
Children:
72e5689
Parents:
a78e3e4
Message:

Fix unicode exception

urllib.quote needs to have "title" encoded as utf-8 to properly deal with URLs having non-ascii chars.

Example:

urllib.quote(u'á')

/usr/lib/python2.7/urllib.py:1288: UnicodeWarning?: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

return .join(map(quoter, s))

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/urllib.py", line 1288, in quote

return .join(map(quoter, s))

KeyError?: u'\xe1'

With encode:

urllib.quote(u'á'.encode('utf-8'))

'%C3%A1'

File:
1 edited

Legend:

Unmodified
Added
Removed
  • plugins/TagFix_Wikipedia.py

    r3f10546 r64b86be  
    8282                    try: 
    8383                        lang, title = tags[wikipediaTag].split(':') 
    84                         json_str = urlread("http://"+lang+".wikipedia.org/w/api.php?action=query&prop=langlinks&titles="+urllib.quote(title)+"&redirects=&lllimit=500&format=json" , 30) 
     84                        json_str = urlread("http://"+lang+".wikipedia.org/w/api.php?action=query&prop=langlinks&titles="+urllib.quote(title.encode('utf-8'))+"&redirects=&lllimit=500&format=json" , 30) 
    8585                        interwiki = json.loads(json_str) 
    8686                        interwiki = dict(map(lambda x: [x["lang"], x["*"]], interwiki["query"]["pages"].values()[0]["langlinks"])) 
Note: See TracChangeset for help on using the changeset viewer.