読者です 読者をやめる 読者になる 読者になる

Yosemiteでmecab-pythonを入れた話

なんか昔Mavericks時代にmecabを入れようとしたら全然入れられなかった気がするけど、Yosemiteにあげてから

brew install mecab
brew install mecab-ipadic

でやると一瞬でインストールできた。brewが賢くなったのかな。

でもその代わり、Pythonバインディングをインストールしようとするとだいぶこけた。ほとんどの記事は

pip install https://mecab.googlecode.com/files/mecab-python-0.996.tar.gz

でインストールしててなんのエラーも出てないらしいけど、僕はsetup.pyが走り始めた時点でエラーが出た。

......
MeCab_wrap.cxx:7669: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7670: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7671: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7672: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7673: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7674: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7675: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7676: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7677: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7678: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7679: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7680: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7681: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7682: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7683: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7684: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7685: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7686: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7687: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7688: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7689: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7690: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7691: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7692: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx: In function ‘void SWIG_InitializeModule(void*)’:
MeCab_wrap.cxx:7807: error: ‘size_t’ was not declared in this scope
MeCab_wrap.cxx:7807: error: expected `;' before ‘i’
MeCab_wrap.cxx:7860: error: ‘i’ was not declared in this scope
MeCab_wrap.cxx:7860: error: ‘struct swig_module_info’ has no member named ‘size’
MeCab_wrap.cxx: In function ‘void SWIG_PropagateClientData()’:
MeCab_wrap.cxx:7960: error: ‘size_t’ was not declared in this scope
MeCab_wrap.cxx:7960: error: expected `;' before ‘i’
MeCab_wrap.cxx:7967: error: ‘i’ was not declared in this scope
MeCab_wrap.cxx:7967: error: ‘struct swig_module_info’ has no member named ‘size’
MeCab_wrap.cxx: At global scope:
MeCab_wrap.cxx:8012: error: ‘Py_ssize_t’ does not name a type
MeCab_wrap.cxx:8064: error: ‘FILE’ has not been declared
MeCab_wrap.cxx: In function ‘int swig_varlink_print(swig_varlinkobject*, int*, int)’:
MeCab_wrap.cxx:8067: error: ‘fprintf’ was not declared in this scope
MeCab_wrap.cxx:8070: error: ‘struct _object’ has no member named ‘ob_refcnt’
MeCab_wrap.cxx: In function ‘void swig_varlink_dealloc(swig_varlinkobject*)’:
MeCab_wrap.cxx:8079: error: ‘free’ was not declared in this scope
MeCab_wrap.cxx: In function ‘PyObject* swig_varlink_getattr(swig_varlinkobject*, char*)’:
MeCab_wrap.cxx:8087: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:8090: error: ‘strcmp’ was not declared in this scope
MeCab_wrap.cxx: In function ‘int swig_varlink_setattr(swig_varlinkobject*, char*, PyObject*)’:
MeCab_wrap.cxx:8107: error: ‘strcmp’ was not declared in this scope
MeCab_wrap.cxx: In function ‘PyTypeObject* swig_varlink_type()’:
MeCab_wrap.cxx:8130: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:8169: error: too many initializers for ‘const PyTypeObject’
MeCab_wrap.cxx:8169: error: invalid conversion from ‘int’ to ‘_typeobject*’
MeCab_wrap.cxx:8169: error: cannot convert ‘char*’ to ‘int (*)(PyObject*, int*, int)’ in initialization
MeCab_wrap.cxx:8169: error: invalid conversion from ‘long unsigned int’ to ‘PyObject* (*)(PyObject*, char*)’
MeCab_wrap.cxx:8169: error: invalid conversion from ‘void (*)(PyObject*)’ to ‘int (*)(PyObject*, PyObject*)’
MeCab_wrap.cxx:8169: error: invalid conversion from ‘int (*)(PyObject*, int*, int)’ to ‘PyObject* (*)(PyObject*)’
MeCab_wrap.cxx:8169: error: cannot convert ‘PyObject* (*)(PyObject*, char*)’ to ‘PyNumberMethods*’ in initialization
MeCab_wrap.cxx:8169: error: cannot convert ‘int (*)(PyObject*, char*, PyObject*)’ to ‘PySequenceMethods*’ in initialization
MeCab_wrap.cxx:8169: error: invalid conversion from ‘PyObject* (*)(PyObject*)’ to ‘long int (*)(PyObject*)’
MeCab_wrap.cxx:8169: error: invalid conversion from ‘PyObject* (*)(PyObject*)’ to ‘long int’
MeCab_wrap.cxx:8169: error: cannot convert ‘char*’ to ‘PyObject* (*)(PyObject*)’ in initialization
MeCab_wrap.cxx: In function ‘PyObject* SWIG_Python_newvarlink()’:
MeCab_wrap.cxx:8185: error: ‘struct _typeobject’ has no member named ‘tp_basicsize’
MeCab_wrap.cxx:8185: error: ‘PyObject_Malloc’ cannot be used as a function
MeCab_wrap.cxx: In function ‘void SWIG_Python_addvarlink(PyObject*, char*, PyObject* (*)(), int (*)(PyObject*))’:
MeCab_wrap.cxx:8195: error: ‘malloc’ was not declared in this scope
MeCab_wrap.cxx:8197: error: ‘size_t’ was not declared in this scope
MeCab_wrap.cxx:8197: error: expected `;' before ‘size’
MeCab_wrap.cxx:8198: error: ‘size’ was not declared in this scope
MeCab_wrap.cxx:8200: error: ‘strncpy’ was not declared in this scope
MeCab_wrap.cxx: In function ‘void SWIG_Python_InstallConstants(PyObject*, swig_const_info*)’:
MeCab_wrap.cxx:8224: error: ‘size_t’ was not declared in this scope
MeCab_wrap.cxx:8224: error: expected `;' before ‘i’
MeCab_wrap.cxx:8225: error: ‘i’ was not declared in this scope
MeCab_wrap.cxx:8228: error: ‘NULL’ was not declared in this scope
MeCab_wrap.cxx:8239: error: ‘struct _object’ has no member named ‘ob_refcnt’
MeCab_wrap.cxx: In function ‘void SWIG_Python_FixMethods(PyMethodDef*, swig_const_info*, swig_type_info**, swig_type_info**)’:
MeCab_wrap.cxx:8253: error: ‘size_t’ was not declared in this scope
MeCab_wrap.cxx:8253: error: expected `;' before ‘i’
MeCab_wrap.cxx:8254: error: ‘i’ was not declared in this scope
MeCab_wrap.cxx:8256: error: ‘strstr’ was not declared in this scope
MeCab_wrap.cxx:8262: error: ‘strlen’ was not declared in this scope
MeCab_wrap.cxx:8262: error: ‘strncmp’ was not declared in this scope
MeCab_wrap.cxx:8270: error: expected `;' before ‘shift’
MeCab_wrap.cxx:8271: error: ‘shift’ was not declared in this scope
MeCab_wrap.cxx:8272: error: expected `;' before ‘ldoc’
MeCab_wrap.cxx:8273: error: expected `;' before ‘lptr’
MeCab_wrap.cxx:8274: error: ‘ldoc’ was not declared in this scope
MeCab_wrap.cxx:8274: error: ‘lptr’ was not declared in this scope
MeCab_wrap.cxx:8274: error: ‘malloc’ was not declared in this scope
MeCab_wrap.cxx:8277: error: ‘strncpy’ was not declared in this scope
MeCab_wrap.cxx: In function ‘void init_MeCab()’:
MeCab_wrap.cxx:8382: error: ‘NULL’ was not declared in this scope
error: command 'gcc-4.2' failed with exit status 1

まずはとりあえずmecab-pythonのパッケージをwgetして、手動でsetup.pyを走らせる。

wget https://mecab.googlecode.com/files/mecab-python-0.996.tar.gz
tar xfvz mecab-python0.996.tar.gz
cd mecab-python0.996
python setup.py build

たぶん上と同じようなビルドエラーが出るんだけど、使っているgccが悪い模様。

CC=/usr/bin/gcc python setup.py build

ただ、今度はbundle1.oが見つからないと。

gcc-4.2 -bundle -undefined dynamic_lookup -L/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.5-x86_64-2.7/MeCab_wrap.o -L/usr/local/Cellar/mecab/0.996/lib -L/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib -lmecab -lstdc++ -o build/lib.macosx-10.5-x86_64-2.7/_MeCab.so
ld: library not found for -lbundle1.o
collect2: ld returned 1 exit status
error: command 'gcc-4.2' failed with exit status 1

これは手動でライブラリパスを通してやる。

locate bundle1.o

でbundle1.oの在り処が見つかるから、LDFLAGSに指定。

CC=/usr/bin/gcc LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/ python setup.py build

これで無事buildが通ったのでinstallできる。

python setup.py install

ここでtestスクリプトを走らせてみる。

$ python test.py
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    import MeCab
  File "/Users/levelfour/Documents/workspace/python/encourager/mecab-python-0.996/MeCab.py", line 26, in <module>
    _MeCab = swig_import_helper()
  File "/Users/levelfour/Documents/workspace/python/encourager/mecab-python-0.996/MeCab.py", line 18, in swig_import_helper
    import _MeCab
ImportError: dlopen(/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/_MeCab.so, 2): Symbol not found: __ZN5MeCab11createModelEPKc
  Referenced from: /Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/_MeCab.so

ぬー。

ちょっとググったら同じようなことで悩んでる人がいるみたい
僕はpyenv上でanaconda-2.0.1を動かしている状況なんだけど、もしかしたらpyenvがいけないのかもしれないと思って

pyenv global system
CC=/usr/bin/gcc LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/ python setup.py build

とやると見事testスクリプトが走るようになった。
いやでもanaconda上でmecabを使いたい。ここでbuildされたファイルを覗いてみた。

$ ls buld/
lib.macosx-10.10-x86_64-2.7  temp.macosx-10.10-x86_64-2.7
lib.macosx-10.5-x86_64-2.7   temp.macosx-10.5-x86_64-2.7

んー。Yosemiteでビルドしているはずなのに10.5向けのファイルが出来上がっている…。
どうやらanaconda-2.0.1でbuildすると10.5のファイルができてしまうらしい。
なにはともあれ、systemのpythonでbuildすると10.10のライブラリができたんだから、コピーすればいいんじゃね?と思ってやってみた。

$ ls build/lib.macosx-10.10-x86_64-2.7/
MeCab.py  _MeCab.so

$ cp build/lib.macosx-10.10-x86_64-2.7/_MeCab.so ~/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/

$ python test.py
0.996
太郎	名詞,固有名詞,人名,名,*,*,太郎,タロウ,タロー
は	助詞,係助詞,*,*,*,*,は,ハ,ワ
この	連体詞,*,*,*,*,*,この,コノ,コノ
本	名詞,一般,*,*,*,*,本,ホン,ホン
を	助詞,格助詞,一般,*,*,*,を,ヲ,ヲ
二	名詞,数,*,*,*,*,二,ニ,ニ
郎	名詞,一般,*,*,*,*,郎,ロウ,ロー
を	助詞,格助詞,一般,*,*,*,を,ヲ,ヲ
見	動詞,自立,*,*,一段,連用形,見る,ミ,ミ
た	助動詞,*,*,*,特殊・タ,基本形,た,タ,タ
女性	名詞,一般,*,*,*,*,女性,ジョセイ,ジョセイ
に	助詞,格助詞,一般,*,*,*,に,ニ,ニ
渡し	動詞,自立,*,*,五段・サ行,連用形,渡す,ワタシ,ワタシ
た	助動詞,*,*,*,特殊・タ,基本形,た,タ,タ
。	記号,句点,*,*,*,*,。,。,。
EOS

 	BOS/EOS,*,*,*,*,*,*,*,*
太郎 	名詞,固有名詞,人名,名,*,*,太郎,タロウ,タロー
は 	助詞,係助詞,*,*,*,*,は,ハ,ワ
この 	連体詞,*,*,*,*,*,この,コノ,コノ
本 	名詞,一般,*,*,*,*,本,ホン,ホン
を 	助詞,格助詞,一般,*,*,*,を,ヲ,ヲ
二 	名詞,数,*,*,*,*,二,ニ,ニ
郎 	名詞,一般,*,*,*,*,郎,ロウ,ロー
を 	助詞,格助詞,一般,*,*,*,を,ヲ,ヲ
見 	動詞,自立,*,*,一段,連用形,見る,ミ,ミ
た 	助動詞,*,*,*,特殊・タ,基本形,た,タ,タ
女性 	名詞,一般,*,*,*,*,女性,ジョセイ,ジョセイ
に 	助詞,格助詞,一般,*,*,*,に,ニ,ニ
渡し 	動詞,自立,*,*,五段・サ行,連用形,渡す,ワタシ,ワタシ
た 	助動詞,*,*,*,特殊・タ,基本形,た,タ,タ
。 	記号,句点,*,*,*,*,。,。,。
 	BOS/EOS,*,*,*,*,*,*,*,*
EOS
EOS
filename: /usr/local/Cellar/mecab/0.996/lib/mecab/dic/ipadic/sys.dic
charset: utf8
size: 392126
type: 0
lsize: 1316
rsize: 1316
version: 102

動いた!

まとめ

  1. CCとLDFLAGSを指定しよう
  2. pyenvが悪さをしてるっぽいから、systemでビルドしたライブラリをコピーしてつっこもう