So how does one test an archaeological predictive model or know how well a predictive model works? If you determine how well a model predicts its own input data, what does it tell you about that data or is it just a circular argument? A predictive model of say the locations of individual stores in a chain of stores would give associations with say population densities and transport routes. But what does that tell you about the individual stores, what they sell or the subtle effects of local planning policies, etc? If you have used the archaeological record (HER) or field walking data or archaeological excavation data to make a model, that model will tell you about the location characteristics of that data only. But does that data represent archaeology or past settlement?
Field walking. The results of the Breckland Survey (Suffolk) suggest a correlation between archaeology and artefact scatters. However, the results of the Shapwick Project suggest that the date of material on the surface may not always be a reliable guide to the date of the buried deposits below (see Gerrad. C. ‘Misplaced faith? Medieval pottery and field walking’ in, Medieval Ceramics volume 21, 1997). Further, work carried out in the Rhone valley in France, specifically in relation to archaeological predictive modelling, suggests that field walking results are not always a good indication of what is below the ground surface (see Verhagan. P & Berger. J, ‘The hidden reserve: predictive modelling of buried archaeological sites in the Tricastin-Valdaine region’ in the 28th conference of the CAA, 2001, BAR 931).
Metal detecting. Responsible metal detectorists report their finds to the county archaeological unit. However, it is very rare that they report areas where they have detected but found nothing. In Norfolk (UK) there is a common perception that most metallic archaeological finds are found in the West of the county. Ergo, majority of Norfolk metal deterctorists work in the West of Norfolk, which strengthens this common view. In other words it is a self-fulfilling situation.
The archaeological record (HER). This tends to be a mixed bag of ad hoc data, subject to various biases such as planning polices (PPG16), the county’s relationship with metal detectorists, the annual budget of the HER officers, etc. As a result it is not easy to model across county or administrative boundaries. The archaeological record is primarily made up of metal detecting, field walking, archaeological excavation, historic buildings and random spot find data.
To produce a good archaeological predictive model, one needs a lot of input data to sample over a small number of categories within any environmental (or social) dataset to obtain sufficient statistical data with which to model with. In other words one cannot sample say 10 pottery sherds over an area with say 15 different soil characteristics and hope to obtain sufficient and reliable data about the relationship between the pottery sherds and the soil! In practice (especially with models that cover a large area) this normally means using the archaeological record (HER) to make the model as there is normally insufficient field walking, metal detecting or archaeological excavation data with which to do so.
So to reword the primary question; what does one use to test an archaeological predictive model? One important factor is if you are considering all archaeology or just archaeological settlements. As it is difficult to define exactly what constitutes the HER, when modelling all archaeology using the HER, you can only use the HER itself to test the model! This bypasses the need to define exactly what the HER is and what it represents (like individual stores within a chain of stores). For example, remove say a random quarter of the HER, produce a predictive model using the remaining three quarters and then see how well that model predicts the removed quarter.
However, if one is modelling archaeological settlement then the situation is more difficult. Not withstanding the doubts noted above, does a scatter of artefacts found by systematic field walking represent a domestic settlement, a temporary encampment or some sort of industrial activity? Further, domestic settlements can be based on an agricultural economy or a pastoral economy, which require very different environments.
Test pitting on a regular grid system has been quoted as a suitable method to test a predictive model. Another way would be to carry out extensive excavations within the high probability areas of a predictive map to see how well it predicts settlement. Of course a handheld GPS unit and a time machine would be an ideal way of testing a predictive model of archaeological settlement. The important thing is that any archaeological predictive model should be tested by an independent means. Thus, does anyone have any practical suggestions or comments on testing archaeological predictive models?
Archaeological predictive modelling started in the 1980s and has grown into a multimillion dollar industry, used almost entirely for cultural heritage management. Interest in predictive modelling was given a considerable boost in 1981 when the US Bureau of Land Management issued an instructional memo, encouraging its use. However, two years later there was a consensus by archaeologists and cultural heritage managers that a lack of scientific rigour and inter-project consistency required a systematic re-evaluation of predictive modelling. This emerged in the form of a landmark publication by Judge. W & Sebastian. L (ed.), ‘Quantifying the Present and Predicting the Past: Theory Method and Application of Archaeological Predictive Modelling’, 1988, US Government Printing Office, America.
Despite the above cautions, some countries and some states in America have continued to rely on archaeological predictive modelling as part of their cultural resource management. For example, one American state has an archaeological predictive model that cost $5 million to produce but estimates that it saves that state $3 million per year by reducing the number of staff required to govern the cultural resources and it has also cut down on administration time (see Madry. S, Cole, M, Gould, S, Resnick, B, Seibel, S, & Wilkerson, M, ‘A GIS based archaeological predictive model and decision support system for the North Carolina department of Transport’, in Mehrer. M & Westcott. K (ed.), GIS and archaeological site location modelling, 2006, Taylor & Francis, London).
In the 1990s, the Dutch government proceeded with a national archaeological predictive model, called the IKAW (Indicatieve Kaart Van Archeologische Waarden), probably because it was estimated that about a third of all archaeology in the Netherlands has been lost since 1950 due to development! The IKAW is now in its second generation and used extensively for cultural heritage management. The resultant map is divided up into high, medium and low areas of predicted archaeology and the policy regarding what action to take with a building development application within that area is defined (see Leusen. M & Kamermans H, ‘Predictive Modelling for Archaeological Heritage Management: A research agenda’, Nederlandse Archeolgische Rapporten 29, 2005, Amersfoort, Holland).
One of the biggest criticisms about archaeological predictive modelling used for cultural heritage management is that the models are self fulfilling. For example, for a building development within a high probability area, an archaeological excavation is likely to be demanded but within a low probability area, only a desk study is likely to be required. Thus, if you only look in a high category area, you will only find archaeology within that area and conversely if you do not look in a low category area, you will never find any archaeology within it!
One can understand the lure of archaeological predictive modelling to cultural heritage managers. It is the Holy Grail of their job! They develop an archaeological predictive map of the area they are responsible for, save a fortune by getting rid of all the regional archaeologists, simply plot any building development application on the predictive map and then issue the appropriate letter stating what archaeological action is required.
Do you agree with the above assessment or do you feel that I have over-simplified or misinterpreted the situation? I would appreciate your thoughts and comments on this sensitive subject.
‘Gain’ is the accepted way of determining (and stating) the effectiveness of an archaeological predictive model, the higher the gain the better the model. In America it is considered that the maximum gain for a predictive model is probably restricted to around 70% due to inherent modelling problems and typically gains range between 50 – 70% (see Ebert. J, ‘The state of art in inductive modelling: seven big mistakes’, in Wescott. K & Brandon. R (ed.), Practical Applications of GIS for Archaeologists: a predictive modelling toolkit, 2000, Taylor & Francis, London). Typically the gains of my models are around 15 – 25% for predicting the model’s own input data and independent test datasets. So the question is why – what am I doing wrong?
As I live in Norfolk (UK), I’m currently developing my predictive modelling skills by modelling Norfolk only, with a view to expanding to the whole of East Anglia (UK) when I am proficient in the art. Oscar Wilde once said that Norfolk is very, very flat and I think that this is a clue – terrain! If one works within a landscape which funnels people to live in specific areas or the environment bars them for living in other areas, then modelling becomes easier. A classic example would be Egypt where the only place people can easily live (in the past or in the present) is next to the Nile (or an oasis) because of the water supply, fertile soil, wild life, etc. Any settlement away from the Nile (or an oasis) would require significant support from elsewhere.
Another factor is ground slope. A classic example is a predictive model in West Virginia (America) that produced high gains in a study area where 90% of the terrain is over 18° and 41% is over 31° (see Lock. G & Harris. T, ‘Enhancing Predictive Archaeological Modelling: Integrating Location, Landscape and Culture’ in Mehrer. M & Westcott. K (ed.), GIS and archaeological site location modelling, 2006, Taylor & Francis, London). Whilst foraging and hunting for food is possible in such a terrain, farming and building permanent settlements on flat ground is severely limited. Consequently, such terrain is ideal for hunter-gatherers who by their migratory life-style leave little evidence of their existence.
Compare these environments with Norfolk (UK); 90% of the county has a ground slope of less than 4° and as a consequence nowhere is very far from a source of open water! For comparison; in 1989 an archaeological predictive model comprising of 120 Km² of the Netherlands (the Regge Valley) was produced. Afterwards, the model was tested by intensely field walking a 2 Km² area within the study area and the settlements discovered were compared with the predictions of the model. The model achieved a high gain for predicting its own input data but a relatively low gain for predicting the field walking discovered settlements (see Brandt. R, Groenewoudt. B & Kvamme. K, ‘An experiment in archaeological site location: modelling in the Netherlands using GIS techniques’, World Archaeology 24: 2, 1992, Routledge, London). The terrain of the Regge Valley is similar to Norfolk and when tested, the Regge Valley model produced similar gains to Norfolk!
Concentrating on just one environmental factor – ground slope, I have examined sixteen published archaeological predictive models and have plotted their stated gain against the average ground slope of each study area (in some cases having to estimate it using various means such as Google Earth). Whilst it is not possible to consider that this plot shows a direct correlation between gain and ground slope (as there are numerous other factors involved) I have noted that the higher the average ground slope of the study area, the higher the gain is likely to be.
So, do I move to America in order to achieve models with higher gains to get my PhD or is it acceptable to carryout archaeological predictive modelling in any terrain? If you have any opinions on this subject I would be pleased to hear from you.
There are two main approaches to archaeological predictive modelling; inductive and deductive. Deductive modelling derives rules from theory or expert knowledge. For example, any settlement would want a close supply of water; therefore there should be more settlements near to open water sources. Inductive modelling derives rules from observations. For example, what is the percentage of known settlements within say 500m of a known open water source? For inductive modelling, the location of past settlement is predicted on the basis of various factors, which broadly fall into two categories; environmental and social. Environmental factors include; soil type, ground slope, etc. Social factors include; existing land ownership, defence, etc. Digital datasets for various environmental factors are readily available, although it is important to remember that they represent the present environment. Having said that, my interest is with the Anglo-Saxon period (410 – 1066AD) and I was surprised by how little overall climatic variation there has been since then (see Lamb. H, ‘Climate: present, past and future’ – volume 2, 1977, Methuen & Co, London). Further, digital soils data (such as fertility, etc) quote ‘base’ properties which do not include for artificial fertilizers or drainage. Thus, suggesting that they are ‘timeless’ properties.
Unless you are modelling the documented past, digital datasets for social factors do not exist and are virtually impossible to make due to a fundamental lack of historical information. One can theorise about social factors but you soon get into all sorts of arguments and counter arguments, which end up with you guessing! For example, during the Anglo-Saxon period the East of England was subjected to raiding from across the North Sea. Therefore, living next to the North Sea or an estuary leading into it would have been precarious. However, in times of peace, these locations make ideal places to settle due to their natural resources and trading links. It all depends upon how large a threat this raiding was perceived at that time.
The degree to which social and environmental factors influenced site location is one of the purposes of academic predictive modelling. Some theoretical settlement models assume the specialization and inter-settlement exchange of local produce (see Aston. M, ‘Interpreting the Landscape’, 1999, Routledge, London). However, majority of settlements would still need to be self sufficient to a reasonable degree; else they would be a significant drain on the resources of their beneficiaries. Thus, environmental factors would still play a significant part in the choice of settlement location. Further, technological advances (such as transport, soil improvement, etc) expand the type of terrain suitable for settlement as it reduces the need for self sufficiency.
So, should archaeological predictive modellers simply ignore all social factors on the assumption that environmental factors over shadow them? If not, then how does one go about determining social datasets to use in archaeological predictive models? I feel that this issue deserves more debate and I would be interested to hear from anyone with a view on this difficult subject.
I’m a PhD student at the University of East Anglia (UK) studying Landscape Archaeology. My research is on the distribution of archaeological artefacts and settlement in East Anglia, concentrating on the differences between early, middle and late Anglo-Saxon periods. I’m interested in the environmental factors which affected the positions of settlement between these historic periods and to search for areas of abnormally high or low concentrations of archaeology. The datasets and techniques I use are identical to those used for archaeological predictive modelling, which some countries and regions incorporate into their cultural heritage management (CHM) strategies. The use of these techniques for academic research is normally not criticised but their use for heritage management is.
Article 5 of the Valletta Agreement (1992) states that; ‘Governments have a duty to see that the interests of archaeology and of development are judiciously weighed against each other’. As a result, some countries have turned to archaeological predictive modelling as part of their cultural heritage management. For inductive predictive modelling, the location of archaeology artefacts and settlement is predicted on the basis of various factors, which broadly fall into two categories; environmental and social. Whilst datasets for environmental factors are available, social datasets are not.
The main criticisms are that predictive models used for CHM are self fulfilling, environmentally deterministic and the datasets they are based on are biased. Exponents of their use for CHM say that such models same time and simplifies administration.
I feel that there should be a place to discuss (and hopefully reach agreement on) these fundamental differences and hence I offer this blog. If your research is in this area or you work in cultural heritage management and use such maps I would like to hear your views on this subject.