Yosemiteでmecab-pythonを入れた話
なんか昔Mavericks時代にmecabを入れようとしたら全然入れられなかった気がするけど、Yosemiteにあげてから
brew install mecab brew install mecab-ipadic
でやると一瞬でインストールできた。brewが賢くなったのかな。
でもその代わり、Pythonバインディングをインストールしようとするとだいぶこけた。ほとんどの記事は
pip install https://mecab.googlecode.com/files/mecab-python-0.996.tar.gz
でインストールしててなんのエラーも出てないらしいけど、僕はsetup.pyが走り始めた時点でエラーが出た。
...... MeCab_wrap.cxx:7669: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7670: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7671: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7672: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7673: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7674: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7675: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7676: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7677: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7678: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7679: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7680: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7681: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7682: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7683: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7684: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7685: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7686: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7687: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7688: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7689: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7690: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7691: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7692: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:7693: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx: In function ‘void SWIG_InitializeModule(void*)’: MeCab_wrap.cxx:7807: error: ‘size_t’ was not declared in this scope MeCab_wrap.cxx:7807: error: expected `;' before ‘i’ MeCab_wrap.cxx:7860: error: ‘i’ was not declared in this scope MeCab_wrap.cxx:7860: error: ‘struct swig_module_info’ has no member named ‘size’ MeCab_wrap.cxx: In function ‘void SWIG_PropagateClientData()’: MeCab_wrap.cxx:7960: error: ‘size_t’ was not declared in this scope MeCab_wrap.cxx:7960: error: expected `;' before ‘i’ MeCab_wrap.cxx:7967: error: ‘i’ was not declared in this scope MeCab_wrap.cxx:7967: error: ‘struct swig_module_info’ has no member named ‘size’ MeCab_wrap.cxx: At global scope: MeCab_wrap.cxx:8012: error: ‘Py_ssize_t’ does not name a type MeCab_wrap.cxx:8064: error: ‘FILE’ has not been declared MeCab_wrap.cxx: In function ‘int swig_varlink_print(swig_varlinkobject*, int*, int)’: MeCab_wrap.cxx:8067: error: ‘fprintf’ was not declared in this scope MeCab_wrap.cxx:8070: error: ‘struct _object’ has no member named ‘ob_refcnt’ MeCab_wrap.cxx: In function ‘void swig_varlink_dealloc(swig_varlinkobject*)’: MeCab_wrap.cxx:8079: error: ‘free’ was not declared in this scope MeCab_wrap.cxx: In function ‘PyObject* swig_varlink_getattr(swig_varlinkobject*, char*)’: MeCab_wrap.cxx:8087: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:8090: error: ‘strcmp’ was not declared in this scope MeCab_wrap.cxx: In function ‘int swig_varlink_setattr(swig_varlinkobject*, char*, PyObject*)’: MeCab_wrap.cxx:8107: error: ‘strcmp’ was not declared in this scope MeCab_wrap.cxx: In function ‘PyTypeObject* swig_varlink_type()’: MeCab_wrap.cxx:8130: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:8169: error: too many initializers for ‘const PyTypeObject’ MeCab_wrap.cxx:8169: error: invalid conversion from ‘int’ to ‘_typeobject*’ MeCab_wrap.cxx:8169: error: cannot convert ‘char*’ to ‘int (*)(PyObject*, int*, int)’ in initialization MeCab_wrap.cxx:8169: error: invalid conversion from ‘long unsigned int’ to ‘PyObject* (*)(PyObject*, char*)’ MeCab_wrap.cxx:8169: error: invalid conversion from ‘void (*)(PyObject*)’ to ‘int (*)(PyObject*, PyObject*)’ MeCab_wrap.cxx:8169: error: invalid conversion from ‘int (*)(PyObject*, int*, int)’ to ‘PyObject* (*)(PyObject*)’ MeCab_wrap.cxx:8169: error: cannot convert ‘PyObject* (*)(PyObject*, char*)’ to ‘PyNumberMethods*’ in initialization MeCab_wrap.cxx:8169: error: cannot convert ‘int (*)(PyObject*, char*, PyObject*)’ to ‘PySequenceMethods*’ in initialization MeCab_wrap.cxx:8169: error: invalid conversion from ‘PyObject* (*)(PyObject*)’ to ‘long int (*)(PyObject*)’ MeCab_wrap.cxx:8169: error: invalid conversion from ‘PyObject* (*)(PyObject*)’ to ‘long int’ MeCab_wrap.cxx:8169: error: cannot convert ‘char*’ to ‘PyObject* (*)(PyObject*)’ in initialization MeCab_wrap.cxx: In function ‘PyObject* SWIG_Python_newvarlink()’: MeCab_wrap.cxx:8185: error: ‘struct _typeobject’ has no member named ‘tp_basicsize’ MeCab_wrap.cxx:8185: error: ‘PyObject_Malloc’ cannot be used as a function MeCab_wrap.cxx: In function ‘void SWIG_Python_addvarlink(PyObject*, char*, PyObject* (*)(), int (*)(PyObject*))’: MeCab_wrap.cxx:8195: error: ‘malloc’ was not declared in this scope MeCab_wrap.cxx:8197: error: ‘size_t’ was not declared in this scope MeCab_wrap.cxx:8197: error: expected `;' before ‘size’ MeCab_wrap.cxx:8198: error: ‘size’ was not declared in this scope MeCab_wrap.cxx:8200: error: ‘strncpy’ was not declared in this scope MeCab_wrap.cxx: In function ‘void SWIG_Python_InstallConstants(PyObject*, swig_const_info*)’: MeCab_wrap.cxx:8224: error: ‘size_t’ was not declared in this scope MeCab_wrap.cxx:8224: error: expected `;' before ‘i’ MeCab_wrap.cxx:8225: error: ‘i’ was not declared in this scope MeCab_wrap.cxx:8228: error: ‘NULL’ was not declared in this scope MeCab_wrap.cxx:8239: error: ‘struct _object’ has no member named ‘ob_refcnt’ MeCab_wrap.cxx: In function ‘void SWIG_Python_FixMethods(PyMethodDef*, swig_const_info*, swig_type_info**, swig_type_info**)’: MeCab_wrap.cxx:8253: error: ‘size_t’ was not declared in this scope MeCab_wrap.cxx:8253: error: expected `;' before ‘i’ MeCab_wrap.cxx:8254: error: ‘i’ was not declared in this scope MeCab_wrap.cxx:8256: error: ‘strstr’ was not declared in this scope MeCab_wrap.cxx:8262: error: ‘strlen’ was not declared in this scope MeCab_wrap.cxx:8262: error: ‘strncmp’ was not declared in this scope MeCab_wrap.cxx:8270: error: expected `;' before ‘shift’ MeCab_wrap.cxx:8271: error: ‘shift’ was not declared in this scope MeCab_wrap.cxx:8272: error: expected `;' before ‘ldoc’ MeCab_wrap.cxx:8273: error: expected `;' before ‘lptr’ MeCab_wrap.cxx:8274: error: ‘ldoc’ was not declared in this scope MeCab_wrap.cxx:8274: error: ‘lptr’ was not declared in this scope MeCab_wrap.cxx:8274: error: ‘malloc’ was not declared in this scope MeCab_wrap.cxx:8277: error: ‘strncpy’ was not declared in this scope MeCab_wrap.cxx: In function ‘void init_MeCab()’: MeCab_wrap.cxx:8382: error: ‘NULL’ was not declared in this scope error: command 'gcc-4.2' failed with exit status 1
まずはとりあえずmecab-pythonのパッケージをwgetして、手動でsetup.pyを走らせる。
wget https://mecab.googlecode.com/files/mecab-python-0.996.tar.gz tar xfvz mecab-python0.996.tar.gz cd mecab-python0.996 python setup.py build
たぶん上と同じようなビルドエラーが出るんだけど、使っているgccが悪い模様。
CC=/usr/bin/gcc python setup.py build
ただ、今度はbundle1.oが見つからないと。
gcc-4.2 -bundle -undefined dynamic_lookup -L/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.5-x86_64-2.7/MeCab_wrap.o -L/usr/local/Cellar/mecab/0.996/lib -L/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib -lmecab -lstdc++ -o build/lib.macosx-10.5-x86_64-2.7/_MeCab.so ld: library not found for -lbundle1.o collect2: ld returned 1 exit status error: command 'gcc-4.2' failed with exit status 1
これは手動でライブラリパスを通してやる。
locate bundle1.o
でbundle1.oの在り処が見つかるから、LDFLAGSに指定。
CC=/usr/bin/gcc LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/ python setup.py build
これで無事buildが通ったのでinstallできる。
python setup.py install
ここでtestスクリプトを走らせてみる。
$ python test.py
Traceback (most recent call last):
File "test.py", line 4, in <module>
import MeCab
File "/Users/levelfour/Documents/workspace/python/encourager/mecab-python-0.996/MeCab.py", line 26, in <module>
_MeCab = swig_import_helper()
File "/Users/levelfour/Documents/workspace/python/encourager/mecab-python-0.996/MeCab.py", line 18, in swig_import_helper
import _MeCab
ImportError: dlopen(/Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/_MeCab.so, 2): Symbol not found: __ZN5MeCab11createModelEPKc
Referenced from: /Users/levelfour/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/_MeCab.so
ぬー。
ちょっとググったら同じようなことで悩んでる人がいるみたい。
僕はpyenv上でanaconda-2.0.1を動かしている状況なんだけど、もしかしたらpyenvがいけないのかもしれないと思って
pyenv global system CC=/usr/bin/gcc LDFLAGS=-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/lib/ python setup.py build
とやると見事testスクリプトが走るようになった。
いやでもanaconda上でmecabを使いたい。ここでbuildされたファイルを覗いてみた。
$ ls buld/ lib.macosx-10.10-x86_64-2.7 temp.macosx-10.10-x86_64-2.7 lib.macosx-10.5-x86_64-2.7 temp.macosx-10.5-x86_64-2.7
んー。Yosemiteでビルドしているはずなのに10.5向けのファイルが出来上がっている…。
どうやらanaconda-2.0.1でbuildすると10.5のファイルができてしまうらしい。
なにはともあれ、systemのpythonでbuildすると10.10のライブラリができたんだから、コピーすればいいんじゃね？と思ってやってみた。
$ ls build/lib.macosx-10.10-x86_64-2.7/ MeCab.py _MeCab.so $ cp build/lib.macosx-10.10-x86_64-2.7/_MeCab.so ~/.pyenv/versions/anaconda-2.0.1/lib/python2.7/site-packages/ $ python test.py 0.996 太郎 名詞,固有名詞,人名,名,*,*,太郎,タロウ,タロー は 助詞,係助詞,*,*,*,*,は,ハ,ワ この 連体詞,*,*,*,*,*,この,コノ,コノ 本 名詞,一般,*,*,*,*,本,ホン,ホン を 助詞,格助詞,一般,*,*,*,を,ヲ,ヲ 二 名詞,数,*,*,*,*,二,ニ,ニ 郎 名詞,一般,*,*,*,*,郎,ロウ,ロー を 助詞,格助詞,一般,*,*,*,を,ヲ,ヲ 見 動詞,自立,*,*,一段,連用形,見る,ミ,ミ た 助動詞,*,*,*,特殊・タ,基本形,た,タ,タ 女性 名詞,一般,*,*,*,*,女性,ジョセイ,ジョセイ に 助詞,格助詞,一般,*,*,*,に,ニ,ニ 渡し 動詞,自立,*,*,五段・サ行,連用形,渡す,ワタシ,ワタシ た 助動詞,*,*,*,特殊・タ,基本形,た,タ,タ 。 記号,句点,*,*,*,*,。,。,。 EOS BOS/EOS,*,*,*,*,*,*,*,* 太郎 名詞,固有名詞,人名,名,*,*,太郎,タロウ,タロー は 助詞,係助詞,*,*,*,*,は,ハ,ワ この 連体詞,*,*,*,*,*,この,コノ,コノ 本 名詞,一般,*,*,*,*,本,ホン,ホン を 助詞,格助詞,一般,*,*,*,を,ヲ,ヲ 二 名詞,数,*,*,*,*,二,ニ,ニ 郎 名詞,一般,*,*,*,*,郎,ロウ,ロー を 助詞,格助詞,一般,*,*,*,を,ヲ,ヲ 見 動詞,自立,*,*,一段,連用形,見る,ミ,ミ た 助動詞,*,*,*,特殊・タ,基本形,た,タ,タ 女性 名詞,一般,*,*,*,*,女性,ジョセイ,ジョセイ に 助詞,格助詞,一般,*,*,*,に,ニ,ニ 渡し 動詞,自立,*,*,五段・サ行,連用形,渡す,ワタシ,ワタシ た 助動詞,*,*,*,特殊・タ,基本形,た,タ,タ 。 記号,句点,*,*,*,*,。,。,。 BOS/EOS,*,*,*,*,*,*,*,* EOS EOS filename: /usr/local/Cellar/mecab/0.996/lib/mecab/dic/ipadic/sys.dic charset: utf8 size: 392126 type: 0 lsize: 1316 rsize: 1316 version: 102
動いた！
まとめ
- CCとLDFLAGSを指定しよう
- pyenvが悪さをしてるっぽいから、systemでビルドしたライブラリをコピーしてつっこもう
