Extract data from a website

The ‘Extract data from a website’ button lets you directly import data from a website into a Silk by simply copying a live URL and pasting it into your Silk dashboard. This service is powered by import.io, an awesome tool that can turn websites into usable data without the need to write any code.

Image title

This feature works best with websites with data that is cleanly structured. Examples of structured pages online are product catalogues, housing listings, ratings, data tables, collections of articles, and so on. The feature doesn’t work on websites with unstructured data.

This option always grabs the latest data from a web page, so you can use it to periodically update a collection of items. It also support pagination: if there is a pagination element (e.g. 'page 1 2 3 4 5'), it'll crawl through them, with a maximum of 5 pages.

To use this feature, go to your Silk’s dashboard, and go to the ‘New Collection’ section. Here, pick the ‘Extract data from a website’ option. Then paste in the URL of the website you wish to extract data from. Hit "Import." That's all it takes.

Here is a video where we import Apple Watch models from the Buy Apple Watch page:

A few things to keep in mind

  • If you get a warning that there are multiple datacards with the same title, pick another tag with the ‘Datacard title’ picker to create nice, unique datacard titles.
  • We import as much data as we can, but you might want to limit the results somewhat. In the importer, use the gear icon and choose ‘Ignore fact’ to ignore any data you don’t need (also shown in the video).
  • We extract the tag names automatically from the website, so they might not always look pretty. Click on the tag names to rename them during the preview process or later after you built your Silk.
  • Same goes for the collection name. You can rename the collection name with the little pencil next to it. You can also change the name after the export.
  • If the import throws an error or the results are too messy to work with, try to extract with import.io directly. You can import the data in to a spreadsheet and clean it up before importing it into Silk.
  • Please make sure you are not republishing copyrighted material.

When everything looks good in the importer window, just hit import, create a visualization out of the imported data, and put it on your home page. Silk will also automatically create some visualizations for your home page. Edit your home page to add some context about the data.

Image title