123ArticleOnline Logo
Welcome to 123ArticleOnline.com!
ALL >> Computer-Programming >> View Article

Straightforward Tips For Efficient Web Crawling Utilizing Selenium Python

By Author: Siyaram ray
Total Articles: 396

Be that as it may, first let me quickly present you with python's selenium module on the off chance that, on the off chance that you are not acquainted with it:

It is really a python authoritative for the API of Selenium Web Drivers. For instance, you will have the capacity to helpfully get to the API of Selenium Web drivers like Firefox, Chrome, and PhantomJS and so on.
Utilizing selenium training in Bangalore this module, you can utilize web driver API to reproduce a wide range of activities that you can perform on a run of the mill Web Browser! i.e. tap on catches of sites, scroll and explore through pages, type something in information boxes, submit frames, use intermediaries, even execute custom Javascript on pages and some more! All these stuff utilizing only python content! Quite cool! Isn’t that so?

Presently, let’s bounce directly to the primary tip:

Tip No 1: Crawl sites without stacking Images

As we are discussing, mechanized contents these contents will run hundreds or thousands time. In this way, consistently (maybe, milliseconds?) check. The majority of the advanced powerful sites have loads of pictures. At the point when a page loads, selenium stacks every one of the components in it including those pictures!

Thus, despite the fact that we don't interface much with those pictures when we are trying site functionalities. Selenium still loads them! The beneficial thing is there are approaches to stack pages without stacking pictures in selenium!

PhantomJS Web Driver Load Web Page without Images:

driver = webdriver.PhantomJS(service_args=['- - stack images=no'])

Chrome Web Driver Load Pages without Images:

chromeOptions = webdriver.ChromeOptions()
prefs = {'profile.managed_default_content_settings.images':2}
chromeOptions.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=chromeOptions)

Tip No 2: Scrape Websites utilizing Disk Cache

Reserving the benefits frequently prompts quicker page loads. In present day internet browser, circle reserving decreases page stacking time amazingly. You can take the upside of this on selenium web driver also!. You should simply set the design before instatement of web driver. This essentially stores every one of the sites resources like css, js in the plate stockpiling for quicker stacking. Accommodating, when you stack various pages of a similar site

Note: Obviously, while computerizing your tests, you can't (and shouldn't) store the benefits that has influences on the information you need to test! For instance, in the event that you are trying a dynamic site where the information is stacked utilizing resources state javascript at that point, circle reserve may even make your tests old!
Tip No 3: Use Javascript for looking over

While communicating with page components exceptionally tapping on catches, if the component you are searching for isn't unmistakable in the view port. Selenium raises an exemption telling that component isn't noticeable. Lets lean toward Javascript for looking over the component into view and, at that point sit tight for a bit utilizing time.sleep() so the parchment impact goes off. Furthermore, at that point trigger the snap… basic!

Tip No 4: Scrolling things in a drop down utilizing Select

Let’s state, you are endeavoring to choose a choice from a select component with colossal number of alternatives state 20+. For this situation, you can't choose the thing in normal way. You should initially find the select component utilizing driver. At that point, discover every one of the alternatives. Channel through every one of them to locate the suitable choice you are searching for. From that point forward, make it noticeable and tap on it! Luckily, selenium has a class called 'Select' which will assist you with doing the above assignment in significantly progressively simpler way!

Tip No 5: Properly shut the driver utilizing quit

It is imperative to legitimately close the driver subsequent to completing the mechanization uniquely, on the off chance that you run your contents occasionally! When you conjure your python computerization content to get things done for you, it utilizes extra assets/forms for the selenium web drivers. After the python content completions execution it doesn't discharges those extra assets on the off chance that you don't instruct it to do as such! One approach to do it is to utilize driver.close().

Be that as it may, driver.close() doesn't generally stop the web driver that is running in foundation. What's more, on the off chance selenium course in Bangalore that you are doing things utilizing numerous tabs of the selenium internet browser then it closes just the present one! Driving others open.

You can utilize driver.quit() rather, as it shuts every one of the tabs of the selenium internet browser . The Selenium Web driver occurrence is additionally gets executed after this! Continuously limit the quantity of solicitations you make to the Web Server. Attempt to diminish it however much as could reasonably be expected.

More About the Author

best institute for selenium training in Bangalore – Infocampus. Infocampus is an excellence training center for selenium courses in Bangalore. At Infocampus, selenium training is focused on practical and real time examples. Contact Infocampus at 8884166608 or visit http://infocampus.co.in/best-selenium-testing-training-center-in-bangalore.html for complete details on selenium training.

Total Views: 227Word Count: 788See All articles From Author

Computer Programming Articles

1. Why Take The Services Of An Iphone Application Development Company
Author: Moon Technolabs

2. Best Cpanel Providers That Offers Whm Cpanel License At Reasonable Prices
Author: Buy cPanel

3. 6 New Technologies Used For Java Developer
Author: infocampus

4. Top Java Web Systems To Use In 2019
Author: infocampus

5. A Brief Look At Custom Healthcare Software Development & Its Future.
Author: pawan yadav

6. Know About Importance Of Java Programming Language
Author: APTRON Noida

7. The Complete Guide To Understanding The Enterprise Application Integration
Author: rishabh Shukla

8. What Java Programmers Should Learn In 2019
Author: infocampus

9. Spring Framework Bean Scopes
Author: Kapil Gupta

10. Master The Art Of Java Se Support Roadmap With These Tips
Author: infocampus

11. How To Use Magento Import Export Extensions To Migrate Product Data?
Author: Shailesh Gajjar

12. A Complete Guide For Advanced Java
Author: infocampus

13. What Rings A Bell When You Hear The Word Java?
Author: infocampus

14. Most Recent Java Technologies And Trends
Author: infocampus

15. How To Disable Firewall On Mac
Author: karen minton

Login To Account
Login Email:
Forgot Password?
New User?
Sign Up Newsletter
Email Address: