python - Scrapy on a JSON response -


i having problem scraping json response in scrapy. getting following error:

typeerror: json object must str, not 'response'

i have tried decode using following importing codecs , decoding response utf-8, isn't working reason. code:

import scrapy import json  class srealitkybuyspiderspider(scrapy.spider):     name = "srealitky_buy"     allowed_domains = ["https://www.sreality.cz/"]     start_urls = ['https://www.sreality.cz/api/cs/v2/estates?category_main_cb=1&category_type_cb=1&per_page=20&region=praha']      def parse(self, response):         jsonresponse = json.loads(response)         print(jsonresponse) 

i tried adding .text response when using json.loads, error "response has no attribute 'text'".

jsonresponse = json.loads(response.text) 

does know wrong?

if @ type(response) scrapy.http.response.text.textresponse

and response.body give b'{"meta_description": "5192 realit v nab\\u00eddce prodej byt\\u016f, .... "collective"}}'

so need use

jsonresponse = json.loads(response.body) print(jsonresponse["meta_description"]) '5192 realit v nabídce prodej bytů, praha. vyberte si novou nemovitost na sreality.cz s hledáním na mapě velkými náhledy fotografií nabízených bytů.' 

edit-1

if above doesn't work because of python version try below

jsonresponse = json.loads(response.body.decode("utf-8")) 

Comments

Popular posts from this blog

ubuntu - PHP script to find files of certain extensions in a directory, returns populated array when run in browser, but empty array when run from terminal -

php - How can i create a user dashboard -

javascript - How to detect toggling of the fullscreen-toolbar in jQuery Mobile? -