Uncategorized

Mar 212019 0 Responses

Talend Introduction & Tutorial to Merge files, having same schema

Extract, Transform, Load (ETL) is the process of extracting data from various data sources, organizing it together, and storing it into a single database for later use like decision making and business insights. Before people used to perform ETL through manual coding in SQL or .NET, but today lots of ETL tools are available that simplify the process. ETL is generally used for data migration, data replication, operational processes, data transformation and data synchronization.

ETL Process

There are many ETL tools available in market both commercial as well as open source like Informatica Power Center, IBM Infosphere Information Server, Oracle Data Integrator, Microsoft SQL Server Integrated Services(SSIS), Ab Initio, Sybase ETL and many more.

ETL has big role in web scraping process. Data scraped from Public websites or other sources are not always in well format or some time it’s messy, ETL tools like Talend and other tools helps to transform the data in required format, validate them, merge them and load it to database like MySQL, NoSQL, sqLite, Oracle and many others or storage target like Amazon S3, FTP, Azure, Dropbox and others. Read More…

Dec 242018 Tagged with , , , 0 Responses

Web Scraping using Content Grabber API

API stands for Application Programming Interface, which is a software intermediary that allows two applications to communicate with each other. The API defines the correct way for a web developer to manage content grabber agent via programming. It is like Program API which uses Remote Procedure Call(RPC) to access the component.

The Content Grabber programming interface (API) provides access to the Content Grabber run-time from your own web/desktop applications For example, If you want to access the result of the Content Grabber Agent in your web application and display it on the dashboard, you can do it easily using the Content Grabber API . The Content Grabber run-time can be distributed with your applications royalty free and does not require the Content Grabber application to be installed on the target computer. The Content Grabber run-time requires .NET version 4.5 or higher.

Content Grabber API Read More…

Dec 192016 Tagged with , , 0 Responses

Total.js Review

Total.js is a free and very powerful web application framework for building Web sites and Web applications using JavaScript, HTML and CSS. The framework is a server-side framework for Node.js. The framework is written in pure JavaScript.

node framework

Total.js framework has very simple logic, a short learning curve and many features for creating rich and scalable web applications. Read More…