欢迎访问 生活随笔!

生活随笔

当前位置: 首页 >

pyhive的安装

发布时间:2023/12/31 63 豆豆
生活随笔 收集整理的这篇文章主要介绍了 pyhive的安装 小编觉得挺不错的,现在分享给大家,帮大家做个参考.

说明名录

  • 连接hive数据库
  • linux下安装pyhive

连接hive数据库

连接hive数据库,许多文章推荐pyhive包,经过小仙女本人多次尝试与周围童鞋交流,发现window系统貌似无法使用pyhive,mmp!!!

所以window系统下,如何连接hive数据库?

  • 果断换包,使用impyla,安装途中遇到各种坑,可以参考下面绝世好文章:
    https://blog.csdn.net/Xiblade/article/details/82318294
    有童鞋推荐使用happybase,没有用过,但感觉是一个可以让人快乐的python包
  • 在电脑硬件还可以情况下,可以选择装一个linux虚拟机
    我选择VMware和Ubantu,方法自行百度,坑很少
  • linux下安装pyhive

    安装上了linux系统,再安装Anoconda,完毕之后打开命令行,依次输入:

    pip install saslpip install thriftpip install thrift-saslpip install pyhive

    然后你就发现安装sasl和thrift-sasl又是各种坑,欸,连接一个hive数据库简直太难了!报错如下:

    Collecting saslDownloading https://files.pythonhosted.org/packages/8e/2c/45dae93d666aea8492678499e0999269b4e55f1829b1e4de5b8204706ad9/sasl-0.2.1.tar.gz Requirement already satisfied: six in ./anaconda3/lib/python3.7/site-packages (from sasl) (1.12.0) Building wheels for collected packages: saslBuilding wheel for sasl (setup.py) ... errorERROR: Complete output from command /home/liuju/anaconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-nc9goial/sasl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-8ytz4zr3 --python-tag cp37:ERROR: running bdist_wheelrunning buildrunning build_pycreating buildcreating build/lib.linux-x86_64-3.7creating build/lib.linux-x86_64-3.7/saslcopying sasl/__init__.py -> build/lib.linux-x86_64-3.7/saslrunning egg_infowriting sasl.egg-info/PKG-INFOwriting dependency_links to sasl.egg-info/dependency_links.txtwriting requirements to sasl.egg-info/requires.txtwriting top-level names to sasl.egg-info/top_level.txtreading manifest file 'sasl.egg-info/SOURCES.txt'reading manifest template 'MANIFEST.in'writing manifest file 'sasl.egg-info/SOURCES.txt'copying sasl/saslwrapper.cpp -> build/lib.linux-x86_64-3.7/saslcopying sasl/saslwrapper.h -> build/lib.linux-x86_64-3.7/saslcopying sasl/saslwrapper.pyx -> build/lib.linux-x86_64-3.7/saslrunning build_extbuilding 'sasl.saslwrapper' extensioncreating build/temp.linux-x86_64-3.7creating build/temp.linux-x86_64-3.7/saslgcc -pthread -B /home/liuju/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isasl -I/home/liuju/anaconda3/include/python3.7m -c sasl/saslwrapper.cpp -o build/temp.linux-x86_64-3.7/sasl/saslwrapper.occ1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++In file included from sasl/saslwrapper.cpp:254:sasl/saslwrapper.h:22:10: fatal error: sasl/sasl.h: No such file or directory#include <sasl/sasl.h>^~~~~~~~~~~~~compilation terminated.error: command 'gcc' failed with exit status 1----------------------------------------ERROR: Failed building wheel for saslRunning setup.py clean for sasl Failed to build sasl Installing collected packages: saslRunning setup.py install for sasl ... errorERROR: Complete output from command /home/liuju/anaconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-nc9goial/sasl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-o68u9xve/install-record.txt --single-version-externally-managed --compile:ERROR: running installrunning buildrunning build_pycreating buildcreating build/lib.linux-x86_64-3.7creating build/lib.linux-x86_64-3.7/saslcopying sasl/__init__.py -> build/lib.linux-x86_64-3.7/saslrunning egg_infowriting sasl.egg-info/PKG-INFOwriting dependency_links to sasl.egg-info/dependency_links.txtwriting requirements to sasl.egg-info/requires.txtwriting top-level names to sasl.egg-info/top_level.txtreading manifest file 'sasl.egg-info/SOURCES.txt'reading manifest template 'MANIFEST.in'writing manifest file 'sasl.egg-info/SOURCES.txt'copying sasl/saslwrapper.cpp -> build/lib.linux-x86_64-3.7/saslcopying sasl/saslwrapper.h -> build/lib.linux-x86_64-3.7/saslcopying sasl/saslwrapper.pyx -> build/lib.linux-x86_64-3.7/saslrunning build_extbuilding 'sasl.saslwrapper' extensioncreating build/temp.linux-x86_64-3.7creating build/temp.linux-x86_64-3.7/saslgcc -pthread -B /home/liuju/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isasl -I/home/liuju/anaconda3/include/python3.7m -c sasl/saslwrapper.cpp -o build/temp.linux-x86_64-3.7/sasl/saslwrapper.occ1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++In file included from sasl/saslwrapper.cpp:254:sasl/saslwrapper.h:22:10: fatal error: sasl/sasl.h: No such file or directory#include <sasl/sasl.h>^~~~~~~~~~~~~compilation terminated.error: command 'gcc' failed with exit status 1---------------------------------------- ERROR: Command "/home/liuju/anaconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-nc9goial/sasl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-o68u9xve/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-nc9goial/sasl/

    其实sasl包的官网上有说明:https://pypi.org/project/sasl/0.1.3/,大概意思就是sasl包依赖gcc

    在安装pyhive相关包之前安装gcc,需要在命令行里输入

    sudo apt-get install python-dev libsasl2-dev gcc



    然后就ojbk了!

    总结

    以上是生活随笔为你收集整理的pyhive的安装的全部内容,希望文章能够帮你解决所遇到的问题。

    如果觉得生活随笔网站内容还不错,欢迎将生活随笔推荐给好友。