爱上python系列------数据集(一):数据集获取api:cbsodata

阳光穿透心脏的1/2处 2022-11-21 09:57 116阅读 0赞

cbsodata是荷兰统计局的数据开放的数据获取的api

平时研究数据科学的时候，都是自己去下载数据集，而cbsodata可以使用api获取数据

使用之前需要安装：

pip install cbsodata

当然conda安装也是可以的：

conda install cbsodata

使用案例：

import pandas as pd
    import cbsodata as cb
    
    name='71509ENG'
    df=pd.DataFrame(cb.get_data(name))
    print(df.head())
    print(cb.get_info(name))

运行结果：

BeurreAlexandreLucas_14         ...           TriompheDeVienne_37
    0                      NaN         ...                           213
    1                      NaN         ...                           212
    2                      NaN         ...                           211
    3                      NaN         ...                           208
    4                      NaN         ...                           211
    
    [5 rows x 41 columns]
    
    {'ID': 0, 'Title': 'Yield and cultivation area apples and pears per region, 1997 - 2017', 'ShortTitle': 'Yield apples and pears, 1997 - 2017', 'Identifier': '71509ENG', 'Summary': 'yield, cultivation area\nfruit variety, region', 'Modified': '2018-03-30T02:00:00', 'ReasonDelivery': 'Discontinued', 'ExplanatoryText': '', 'Language': 'en', 'Catalog': 'CBS', 'Frequency': 'Discontinued', 'Period': '2017', 'ShortDescription': '\nThis table provides information on the harvest of apples and pears in the Netherlands. It concerns the harvest of apples and pears and the  corresponding cultivation area (broken down by variety and fruit cultivation area).\nFrom 2016 it is mandatory for growers to participate in the survey. The response is therefore higher than in previous years, which can lead to breaks in the trend.\n\nData available from 1997 to 2017.\n\nStatus of the figures:\nThe figures for the years 1997 to 2016 are final, and the figures for 2017 are provisional. Due to the discontinuation, the provisional figures will not become definite.\n\nChanges as of 12 April 2019: \nNone, this table has been discontinued.\n\nWhen will new figures be published?\nNot. This table has been discontinued.\n', 'Description': "CONTENTS \r\n\r\n1. General information \r\n2. Definitions and explanation of symbols \r\n3. Links to relevant tables and articles \r\n4. Sources and methods \r\n5. Further information \r\n\r\n1. GENERAL INFORMATION \r\n\r\nThis table provides information on the harvest of apples and pears in the Netherlands. It concerns the harvest of apples and pears and the  corresponding cultivation area (broken down by variety and fruit cultivation area).\r\nFrom 2016 it is mandatory for growers to participate in the survey. The response is therefore higher than in previous years, which can lead to breaks in the trend.\r\n\r\nData available from 1997 to 2017.\r\n\r\nStatus of the figures:\r\nThe figures for the years 1997 to 2016 are final, and the figures for 2017 are provisional. Due to the discontinuation, the provisional figures will not become definite.\r\n\r\nChanges as of 12 April 2019: \r\nNone, this table has been discontinued.\r\n\r\nWhen will new figures be published?\r\nNot. This table has been discontinued.\r\n\r\n2. DEFINITIONS AND EXPLANATION OF SYMBOLS\r\n\r\nDefinitions:\r\n\r\nYield\r\nApple and pear harvest, in million kg.\r\n\r\nCultivation area\r\nThe area of apple and pear orchards, in hectares.\r\n\r\nFruit cultivation areas\r\nClassification into four specific fruit cultivation areas. The composition of the cultivation areas is such that within one region, the same cultivation and market conditions apply.\r\n\r\nApple varieties are: Cox's Orange Pippin, Delbarestivale, Elstar, Golden Delicious, Jonagold, Jonagored, Junami, Kanzi, Rode Boskoop, Rubens.\r\nRare apple varieties, such as Gala and Braeburn, are presented in the category 'Other apple varieties'.\r\nFrom 2016 the variety Rubens (incl. Civni) is part of the category 'Other apple varieties'. \r\n\r\nThe pear varieties are: Beurré Alexandre Lucas, Conference, Doyenné du Comice, Cooking pears, Triomphe de Vienne. \r\nRare pear varieties, such as Beurré Hardy en Clapp's Favourite, are presented in the category 'Other pear varieties'.\r\n\r\nExplanation of symbols:\r\n\r\nEmpty cell\t: figure not applicable\r\n. \t\t: figure is unknown, insufficiently reliable or confidential\r\n*\t\t: provisional figure \r\n**\t\t: revised provisional figure\r\n\r\n3. LINKS TO RELEVANT TABLES AND ARTICLES\t\r\n\r\nRelevant articles:\r\n\r\nMore information about agriculture can be found on: \r\n<a href='https://www.cbs.nl/en-gb/economy/agriculture'>Theme page Agriculture</a>\t\r\n\r\n4. SOURCES AND METHODS\t\r\n\r\nSources:\r\n\r\nIn spring, the annual agricultural census is held and the total apple and pear cultivation areas in the Netherlands are surveyed. On the basis of this information, Statistics Netherlands takes a sample survey to estimate the apple and pear harvest broken down by variety and fruit cultivation area. \r\nThe results of the sample survey are subsequently discussed by an expert panel.\r\n\r\n5. FURTHER INFORMATION\r\n\r\n<a href='https://www.cbs.nl/en-gb/about-us/contact/infoservice'>Infoservice</a>\r\n\r\nCopyright (c) Statistics Netherlands, The Hague/Heerlen\r\nReproduction is permitted, provided Statistics Netherlands is quoted as the source. ", 'DefaultPresentation': 'graphType=table', 'DefaultSelection': "$filter=((FruitFarmingRegions eq '1') or (FruitFarmingRegions eq '2') or (FruitFarmingRegions eq '4') or (FruitFarmingRegions eq '3') or (FruitFarmingRegions eq '5')) and ((Periods eq '1997JJ00') or (Periods eq '2012JJ00') or (Periods eq '2013JJ00') or (Periods eq '2016JJ00'))&$select=FruitFarmingRegions, Periods, TotalAppleVarieties_1, CoxSOrangePippin_2, DelbarestivaleDelcorf_3, Elstar_4, GoldenDelicious_5, Jonagold_6, Jonagored_7, RodeBoskoopRennetApple_10, OtherAppleVarieties_12, TotalPearVarieties_13, Conference_15, DoyenneDuComice_16, CookingPears_17, TriompheDeVienne_18, OtherPearVarieties_19, TotalAppleVarieties_20, CoxSOrangePippin_21, DelbarestivaleDelcorf_22, Elstar_23, GoldenDelicious_24, Jonagold_25, Jonagored_26, RodeBoskoopRennetApple_29, OtherAppleVarieties_31, TotalPearVarieties_32, Conference_34, DoyenneDuComice_35, CookingPears_36, TriompheDeVienne_37, OtherPearVarieties_38&k=Periods,FruitFarmingRegions&r=Topics", 'GraphTypes': 'Table,Bar,Line', 'OutputStatus': 'Discontinued', 'Source': 'CBS.', 'MetaDataModified': '2019-04-12T02:00:00', 'SearchPriority': '1'}

上面的那个name,需要自己获取，我找了很久还是没有找到api可以检索或者查找name的方式

cb.catalog
    cb.get_data
    cb.get_table_list 
    cb.cbsodata3      
    cb.get_info       
    cb.options        
    cb.download_data 
    cb.get_meta

不过，找到[个R版的][R]可以获取name

[R]: https://github.com/edwindj/cbsodataR