How to solve error while launching scrapy shell?
How to solve error while launching scrapy shell?
when i executed scrapy shell 'https://scrapy.org' the following error occured, i followed the instructions from the scrapy documentation https://doc.scrapy.org/en/latest/topics/shell.html#launch-the-shell
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 150, in execute
_run_print_help(parser, _run_command, cmd, args, opts)
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 90, in _run_print_help
func(*a, **kw)
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 157, in _run_command
cmd.run(args, opts)
File "/usr/local/lib/python2.7/dist-packages/scrapy/commands/shell.py", line 65, in run
crawler = self.crawler_process._create_crawler(spidercls)
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 203, in _create_crawler
return Crawler(spidercls, self.settings)
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 55, in __init__
self.extensions = ExtensionManager.from_crawler(self)
File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 58, in from_crawler
return cls.from_settings(crawler.settings, crawler)
File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 34, in from_settings
mwcls = load_object(clspath)
File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 44, in load_object
mod = import_module(module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/usr/local/lib/python2.7/dist-packages/scrapy/extensions/memusage.py", line 16, in <module>
from scrapy.mail import MailSender
File "/usr/local/lib/python2.7/dist-packages/scrapy/mail.py", line 25, in <module>
from twisted.internet import defer, reactor, ssl
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/ssl.py", line 230, in <module>
from twisted.internet._sslverify import (
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/_sslverify.py", line 15, in <module>
from OpenSSL._util import lib as pyOpenSSLlib
ImportError: No module named _util
1 Answer
1
It's a problem in scrapy installation. Maybe this can help:
sudo pip install pyopenssl --user --upgrade
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
