Assessment of accuracy
The CEH Land Cover® plus: Crops map was validated using data collated by the Rural Payments Agency, Rural Payments Wales and Scottish Government Rural Payments and Services. These datasets comprise a record of all land registered for agricultural use and are submitted annually by farmers. Crops are assigned to unique land parcels using the Land Parcel Identification System (LPIS). The crop map was intersected with LPIS parcels, excluding those where a one-to-one match between the Land Cover® plus: Crops map polygon and the LPIS parcels was not clear. We also excluded LPIS parcels where multiple crops had been registered in the same year, because the locations of each different crop within the parcel are not recorded and so cannot be compared to corresponding Land Cover® plus: Crops polygons with multiple crops.
For Land Cover® plus: Crops 2015, after these exclusions, the final validation dataset consisted of crop data on around 250,000 fields covering 1.3 million hectares of agricultural land in Scotland, Wales and England. For Land Cover® plus: Crops 2016 the final validation dataset consisted of crop data on around 860,000 fields covering 0.9 million hectares of agricultural land in England.
Total accuracy was assessed as overall percentage correct and the kappa statistics, which takes account of expected accuracy under random assignment. Overall percentage correct were 95% for 2015 and 87% for 2016. In both 2015 and 2016 kappa was around 0.82 (largely because of the large number of grass fields included in the 2015 data lead to a higher expected accuracy under random assignment).
Accuracy per crop can be measured in two ways: the ability of Land Cover® plus: Crops to correctly detect known crops (% detected) and the proportion of crop assignments in Land Cover® plus: Crops which are correct (% correct). In many cases these two measures of accuracy were similar. For Land Cover® plus: Crops 2015 and 2016 accuracy in terms of both detection and correct identification was uniformly high for the UK’s dominant crops: oilseed rape (> 89%), grass (> 94%) and winter cereals (> 78%).
The combined results of the validation for Land Cover® plus: Crops 2015 and 2016 is shown below:
Some apparent inaccuracy is to be expected due to differences in the crop classifications. For example:
- The Land Cover® plus: Crops map 'Potatoes' class only includes main crop potatoes, while early crop potatoes are assigned to the ‘Other’ class. The validation data does not differentiate between these two.
- Land Cover® plus: Crops classifies spring vs. winter cereals based by sowing date, whereas validation data does so by variety.
- For many crops in the 'other' class it is hard to determine the correct Land Cover® plus: Crops class in order to assess accuracy. For example, orchards and perennial crops might be expected to be classified as either 'other' or 'grass' depending on precise crop and management
- Anomalous fields (eg. poor establishment, crop failure, whole crop harvest) are likely to be classified as the 'other' class.
Further expected misclassifications are listed in the metadata - see the attached spreadsheet [xls].
In some cases, agricultural land may be missing from Land Cover® plus: Crops due to its dependence on the LCM2007 to identify agricultural land for classification. Where this was were inaccurate in the original classification, Land Cover® plus: Crops may be missing data or attempting to assign a crop to parcels which are not agricultural. This issue has been improved in later years by including additional parcels which were classified as rough grassland in LCM2007 and using Ordnance Survey data to filter out non-agricultural land parcels.
CEH Land Cover® plus: Crops 2015
For the first CEH Land Cover® plus: Crops map, in 2015, the concept and approach was tested. The dataset, which covers all of Great Britain, was only finalised where satellite imagery and ground-truthing data of a sufficient quality and quantity were available. Therefore, the quality of the 2015 CEH Land Cover® plus: Crops data varies according to geographical location, with only those parts of the dataset considered to be of an acceptable quality available for licensing. These areas are coloured red in the image below (ie. data for Scotland, Wales and the area from the Welsh borders, through the Midlands to Lincolnshire).
The figures below, which have been independently validated, represent indicative accuracies for CEH Land Cover® plus: Crops map 2015 in just Wales and Scotland:
- Oilseed Rape (97%, or 97 out of every 100 fields correctly mapped)
- Winter Wheat/Winter Barley/Spring Barley (90%, or 9 out of every 10 fields correctly mapped)
- Field Beans/Sugar Beet/Potatoes/Maize (80%, or 8 out of every 10 fields correctly mapped)
- Improved Grass (95%, or 19 out of every 20 fields correctly mapped).
CEH Land Cover® plus: Crops 2016, 2017 and 2018
Independent accuracy figures for CEH Land Cover® plus: Crops 2017 and 2018 are unavailable, but the methodology for 2016, 2017 and 2018 was the same so accuracy is broadly equal.
(QA text by John Redhead, 3/10/18).