Authority Levels

At the Berglund Center for Internet Studies and the Matsushita Center for Electronic Learning at Pacific University we have long searched for a means to indicate the level of authority of the electronic materials we post.  We are well-established centers with well over five million hits per year into our servers. We publish several journals of significance and influence, such as the Journal of the Association for History and Computing (JAHC), Interface on the Internet, and E-AsPac. However, we also publish many projects done by graduate and undergraduate students.  Between these two poles we publish conference papers and occasional papers done by members of our audiences. These materials, then, are created for different purposes and to different review standards.

We know that our audience often selects these materials indiscriminately, based more on their immediate research needs than on the reliability or authority of the materials. In an effort to further educate our audiences, and out of our desire to help establish standards and practices for electronic materials, we have decided to begin “stamping” our materials posted from fall of 2004 with seals indicating their Authority Level.

By Authority Level we do not mean to indicate “good” and “bad” but rather authoritative and non-authoritative or those with higher levels of trust and those with lower levels. Many important works have been created by writers who did not undergo peer-review, did not possess a terminal degree in the field, and who wrote badly with scant attention to scholarly niceties.  However, such materials should best be judged by other scholars, or by the test of proving useful or provocative over time.  They cannot be trusted at the first reading; they do not speak with great authority.

By applying a seal attesting to Authority Level, we are stating that insofar as we can determine, the reader can “trust” the conclusions of the pieces posted with the indicated level of confidence. In short, pieces at lower levels may be excellent, but we can’t determine that they are, and the reader should have a lower initial level of confidence in them.

These levels are explained below, but it should be noted that invariably some materials are in grey areas between categories.

Authority Level 5
Level 5 materials have to pass a number of tests and represent the highest possible level of trust or authority. They are equivalent to materials published in paper form by scholarly journals or presses. They could be placed in a library and should prove useful for some length of time.

They have the following characteristics, but no piece, of course, will necessarily have all of them:

• Materials certified as Authority Level 5 have been reviewed by scholars in the appropriate field following practices long utilized in scholarly journals and other refereed publications. That is, they have been read closely by one or more established professionals comfortable with the topic area of the piece.  The review process was “double-blind.” That is, neither the authors nor the reviewers were aware of each other’s identities. 
• In the judgments of these readers, the piece makes a contribution to the topic being discussed; that is, they are not merely a restatement of existing scholarly opinions.
• The materials have both citations and bibliographies sufficient to permit readers to retrace the author’s research steps so as to form their own opinion as to the strength and weaknesses of the pieces when measured against the set of evidence on which they were constructed.
• The author should discuss methodology as well as evidence: how do they know what they know?
• The articles show an awareness of the current state of the topic by referencing or discussing recent scholarship in both important books and articles. The author should probably indicate areas where there are differences of opinion among authorities.
• The author of the piece is known, can be contacted to discuss or defend his or her positions, and themselves have some specialized education, training, or experience relative to their topic.
• The piece is well written and organized and has a minimum of spelling, grammatical or formatting errors, showing the qualities of mind necessary to good research, thought, and writing.
• Such publications would usually include research done in the language(s) of the subject field as appropriate.
• Limitations of Level 5 materials:  Despite these strict standards, even the best research and writing has areas of relative weakness, and scholars in the field will often disagree about each others’ positions. The more “cutting edge” a piece, the more likely it is to be controversial. For the average reader, however, these controversies are likely to be unimportant ones.
  

Authority Level 4
Materials certified as Level 4 are not different so much in quality as in degree from Level 5 materials.  They might have the following characteristics:

• The level of authority of the writer may not be the highest possible in the field of study and the process of review may be less thorough.
• The pieces have been read by somebody with some expertise in the field and in effect are certified as free from egregious errors of fact or important omissions.
• The authors are known, can be contacted, and have some relative level of expertise in the field about which they are writing.
• The materials used in the research are obvious if not always strictly cited. A reader should be able, then, to roughly reconstruct the author’s research paths.
• The author’s methodology should be at least strongly implied if not obvious.
• The pieces should be based on at least some original materials (those written by eyewitnesses or participants with direct knowledge of the events or issues discussed) rather than entirely in secondary materials (those written from analyses of original materials).
• There should be a bibliography.
• The author should demonstrate some expertise in dealing with scholarly controversies and materials.
• Limitations of Level 4 materials: These materials can be used with confidence, but it may well be that there are better materials to be found on these same topics.
  
  

Authority Level 3
Level 3 materials are often the result of research projects done by inexperienced authors, but guided by experienced ones. These might include extensive undergraduate research projects. They have the following characteristics:

• Level 3 materials should be free from errors of fact. If some of their conclusions are controversial ones, these should be held by at least some authorities in the field of study.
• Level 3 materials should depend to a considerable extent upon published and easily available materials so that readers can themselves explore the topic further with some confidence.
• The basis of conclusions drawn in level 3 materials should be clear and rooted in research, not in mere opinion or prejudice.
• Level 3 authors should be known, and can be contacted to discuss their works.
• The works are dated as to time of origin and any updates that occurred,
• Limitations of level 3 materials: Level 3 materials may provide an adequate beginning for serious research, but should not be the sole foundation of such research. They might well provide a good orientation to the topic and suggest additional research paths, but this is their major strength.
  

Authority Level 2
Authority Level 2 materials have serious limitations and it is our intention not to publish such materials.  For our purposes, however, they might be known by the following characteristics:

• Writing is poorly organized, there are spelling and grammatical errors, all indicators of superficial thinking and research.
• Materials are inadequately cited and there is no useful bibliography so that it is impossible to retrace the author’s steps.
• The nature of the evidence relied upon is not clear.
• The author in no way relies upon any obvious authoritative sources or materials.
• The author is not known or cannot be contacted.
• The materials are not dated.
• Utility of Level 2 materials: It may be that serious scholars or researchers could utilize such materials, but most often as the raw material of their own studies.  For example, scholars studying popular culture might find blog entries on current films or music or politics useful but would be unlikely to cite them to support their own positions.
  

Authority Level 1
Again, we will never post such materials, but for analytical purposes we would define them as having the following characteristics:

• A hypothetical “reasonable reader” would find them indicative of non-logical thought processes.
• Writing would be confusing or ambiguous.
• Organization would be so poor as to leave the reader lost and confused; we do not know why paragraphs are arranged as they are.
• There would be no indication of author and no attempt to establish authority.
• Utility of Level 1 materials: Like Level 2 materials, they might be useful for some scholarly projects, but not to convey information or analysis.
   

Please note that the standards for Level of Trust for posted pieces vary from those appropriate to electronic sites or clusters per se.  For a discussion of the qualities of  “good” sites see: http://mcel.pacificu.edu/history/jahci1/epee.htm