PyCharm IDE breakpoints don't work in subprocess call

PyCharm IDE breakpoints don't work in subprocess call



I have a Flask project running with a subprocess call to a Scrapy spider:


class Utilities(object):

@staticmethod
def scrape(inputs):
job_id = str(uuid.uuid4())
project_folder = os.path.abspath(os.path.dirname(__file__))
subprocess.call(['scrapy', 'crawl', "ExampleCrawler", "-a", "inputs=" + str(inputs), "-s", "JOB_ID=" + job_id],
cwd="%s/scraper" % project_folder)
return job_id



Even though I have 'Attach to subprocess automatically while debugging' enabled in the projects' Python debugger, the breakpoints inside the spider won't work. The first breakpoint which works again would be one on return job_id.


return job_id



Here is a part of the code from the spider where I expect the breakpoints to work:


from scrapy.http import FormRequest
from scrapy.spiders import Spider
from scrapy.loader import ItemLoader
from Handelsregister_Scraper.scraper.items import Product
import re


class ExampleCrawler(Spider):
name = "ExampleCrawler"

def __init__(self, inputs='', *args, **kwargs):
super(ExampleCrawler, self).__init__(*args, **kwargs)
self.start_urls = ['https://www.example-link.com']
self.inputs = inputs

def parse(self, response):
yield FormRequest(self.start_urls[0], callback=self.parse_list_elements, formdata=self.inputs)



I can't find any solution to this other than enabling said option which i did.



Any suggestions how to get breakpoints inside the spider work?




2 Answers
2



debugger doesn't work because is not a subprocess, but an external call. see this answer for possible workarounds.



Try configuring this python running configuration in "Edit Configurations". After that, run in debug mode.



Example



Thanks for contributing an answer to Stack Overflow!



But avoid …



To learn more, see our tips on writing great answers.



Required, but never shown



Required, but never shown




By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

PHP code is not being executed, instead code shows on the page

Image
266 58 I'm trying to execute some PHP code on a project (using Dreamweaver) but the code isn't being run. When I check the source code, the PHP code appears as HTML tags (I can see it in the source code). Apache is running properly (I'm working with XAMPP), the PHP pages are being opened properly but the PHP code isn't being executed. Does someone have a suggestion about what is happening? Note: The file is already named as filename.php Edit: The Code..: <? include_once("/code/configs.php"); ?> php apache share | improve this question edited Mar 22 '17 at 16:16 Taryn ♦ 191k 47 291 354 asked Feb 25 '11 at 19:06 Gui Gui 3,652 10 34 51 38 Are you using short tags <? instead of <?php ? – Dan Grossman Feb 25 '11 at 19:09 2 Yes, i'm using <? ... – Gui Feb 25 '11 at 19:12 22 Don't EVER use short tags. ( <? ). They are depre...
Read more

Administrative divisions of China

Image
Clash Royale CLAN TAG #URR8PPP For other uses, see Administrative divisions of China (disambiguation). This article is about the administrative divisions in the People's Republic of China. For those of the Republic of China commonly known as Taiwan, see Administrative divisions of the Republic of China. This article is part of a series on Administrative divisions of China Provincial level (1st) Municipalities Provinces Autonomous regions Special administrative regions Sub-provincial level Sub-provincial cities Sub-provincial autonomous prefectures Sub-provincial city districts Prefectural level (2nd) Prefectural cities Autonomous prefectures Leagues Prefectures (abolishing) Sub-prefectural-level Sub-prefectural cities Provincial-controlled cities Provincial-controlled counties Provincial-controlled districts County level (3rd) Counties Autonomous counties County-level cities Districts Ethnic districts Banners Autonomous banners Shennongjia Forestry District Liuzhi Special District...
Read more

Cardinality of the set of algorithms

Image
15 3 $begingroup$ Someone in a discussion brought up that (he reckons) there can be at least continuum number of strategies to approach a specific problem. The specific problem was trading strategies (not algorithms but strategies) but I think thats beside the point for my question. This got me thinking about the cardinality of the set of algorithms. I have been searching around a bit but have come up with nothing. I've been thinking that, since turing machines operate with a finite set of alphabet and the tape has to be indexable thus countable, it's impossible to have uncountable number of algorithms. My set theory is admittedly rusty so I am not certain at all my reasoning is valid and I probably wouldn't be able to prove it, but it's an interesting thought. What is the cardinality of the set of algorithms? algorithms turing-machines combinatorics share | cite | improve this question edited Aug 28 '18 at 13:59 Raphael ♦ 57.8k 24...
Read more