After writing here a (usually) acerbic expression on bad research (short summary – it is the one appealing purely to authority status) it is time to make a statement on what I (subjectively) consider quality research method. Today in historical studies we move away from blind repetition of chronicles to a task akin to military intelligence or proper science – piecing together the existing information, filtering out what we consider to be the noise and finding an explanation that fits the best with the remaining data. Here are a few basic principles I consider to be of great importance when doing so:
1. Commonality – there are can not be too many exceptions. We are herd animals. In every task, in every acquisition we have to rely on “established practice”, and operate with resources typical for our time and place. Thus most items and events represent general trends rather than unique entities. If someone tells you that a country legalized slavery in the 19th century, or went to gold standard in the 20th century, or suddenly mobilized for a battlefield 10 times more soldiers compared to any other battle it had within the same century – it is worthy of hard look into the facts they base this assessment on. The exceptions are far too often claimed, and far too seldom occur in practice.
2. Proportionality – a great event should generate proportionally great informational response – and vice versa, a single minute piece of evidence can not be considered proof of exceptional change. For example, you’ll find that often the largest, exceptional battles are reconstructed on the basis of much fewer contemporary sources compared to more “typical” period engagements – as it is much easier to find a singular wild statement (millions of dead, the battle lasted three days, rivers of blood etc. etc.), than a stream of self-collaborating, consistent contemporary evidence of such carnage. Therefore, some of supposedly the greatest medieval battles, such as Kulikovo field, were in fact of such modest significance, they did not merit much coverage in contemporary documents. Which opened the door to their reinterpretation based on often centuries (!) later poems (!!).
3. Consistency. If we use something as evidence than similar sources have to be used as evidence in other cases as well. For example, one often finds a particular reference from a particular book or document being cited over and over (for example, famous early genealogy of Masamune), but the rest of the document or similar documents from the same period never being referenced at all. The reason – the information they provide is inconsistent with other sources. Similarly, you’ll find that the methods used by linguists to estimate which language was spoken by such ancient peoples as Scythians would yield grotesque results, if used with later nations, where the predictions of such methods can actually be verified. Often we have to use what we can, but we have to be mindful – it can not serve as definitive proof, if we have to cherry pick 5% of the document, and just forget the rest.
4. Exact repetition is not new evidence. If you find 5 books offering the same casualty estimate, i.e. solution to a very complex, non-linear problem with a lot of data noise and unknowns, it is not because 5 authors solved it and arrived to the same solution. It is because they all copied the 6th, earlier book.
5. Emotionality – the more passionate and colorful the description is, the more likely it is wrong. Rivers of dead, a blade that slew thousands in a day, they called him Demon of Baghdad, a secret technique of secret masters, samurai on bushido, absolute ninja steel – all those are things of teenage mind, and likely a product of teenage excitement, not a well perceived reality. There is a strong anti-correlation between pathos and uncontrolled emotion on one side and intellect on the other.
6. Moralism. Most people today have a big problem with slaughtering children, legislating slavery and annexing land. Thus 90% of historical research is about “did they really do it” or “where they justified in doing it”. What research should be about is something more physics like: does force A leads to B? How fast does this process unfold? If you see a moralizing book, just reading the introduction is enough – the rest of 300 pages will just repeatedly confirm the author’s initial hypothesis. And this always happens, unless you are willing to honestly challenge your theories. Which leads me to the next point:
7. Do not be too much or too little loyal to your hypothesis. Nothing can be absolutely certainly proven with regards of history, but we can identify the most likely scenario. In doing so, one has to “filter” the evidence, and is often guided by some theoretical hypotheses. But if you encounter a quality new data, it has to be adjusted. Also, if you encounter some poor quality, noisy new evidence contradicting it – the hypothesis can not be immediately discarded. This also puts limits on arguing in public space – if the purpose if achieving “victory” in a debate, it is pointless to participate in one, as there is always some noisy data that can be found to “prove” almost any point of view.