Review Criteria
REVIEW CRITERIA: PAPERS & POSTERS
Originality & Creativity (15%)
Does the submission contain new and significant information adequate to justify its inclusion? Are the ideas presented in the submission innovative or creative?
10 - Original and innovative work.
08 - Original work but not particularly innovative.
06 - Contains new, significant information, but mainly reproduces earlier research.
04 - Contains some new information, but it is not significant or sufficient to justify its inclusion.
02 - Lacks new or significant information.
Theoretical framework and/or methodology (15%)
Does the submission use or build upon an appropriate foundation of theories or concepts? If the submission presents empirical research, are the methods, including data or material collection and analysis, appropriate and sound? Are the methods in line with the presented theoretical assumptions? Does the paper present a new theory or new concepts? Are the theoretical or empirical research findings accurately and adequately described?
10 - Excellent use of theory and/or research methods. Applies research methods and/or develops theory in an innovative way. Results are clearly presented and explained in a meaningful and original way.
08 - Skillfully builds on prior work. Significant use of theory and valid research methods. Results are clearly presented and explained.
06 - Relevant prior work is mentioned. Appropriate use of theory and/or research methods.
04 - Important, relevant prior work is missing. Presents some description of a theoretical framework and/or research methods, but important details are missing or there are issues with the data collection and/or analysis.
02 - Significant shortcomings in the theoretical framework or methodology.
Implications for research, practice, and/or society (15%)
Does the submission identify implications for research, practice, and/or society? For example, does it contribute to our understanding of the topic under discussion or to a particular body of knowledge? Or does it discuss how the results, or its insights can be used in teaching, to influence public policy, or to improve quality of life? Or does it consider a potential commercial impact or potential negative consequences? Are the implications mentioned in the submission significant and consistent with the research presented in the submission?
10 - The implications are highly significant and clearly based on the results presented.
08 - The implications are likely to be significant and based on the results presented.
06 - A valid implication is discussed.
04 - No meaningful implications are described, but there are significant implications that the authors should include in a revision.
02 - No implications are described, and it is unlikely that there would be significant implications.
Clarity of submission (15%)
Is the submission well-written?
10 - Very well written, easy to follow, with no or few errors.
08 – Well written, but the clarity could be improved.
06 - The major points are clear, but there are significant shortcomings in the clarity.
04 - The submission is difficult to understand and/or misses important parts.
02 - The submission is very difficult/impossible to understand and/or misses key parts.
Potential for engaging conference participants (15%)
Please estimate the interest you think this contribution will generate. Will there be general excitement and enthusiasm for the submission (i.e., great interest)? Will there be a portion of attendees who will come to hear about it or read the paper in the conference proceedings (i.e., moderate interest)? Or will only a handful of attendees want to hear about it (i.e., low interest)?
10 - Very high potential
08 - High potential
06 - Average potential
04 - Low potential
02 - Very low/no potential
Overall recommendation (25%)
Should this submission be considered for inclusion in the Annual Meeting? How important is this work? Will people take away something useful from this contribution? Please think carefully about your overall recommendation. Should the submission be accepted or rejected?
10 - Strong accept (a very high-quality contribution)
08 - Accept (a good, solid contribution)
06 - Undecided
04 - Weak reject
02 - Strong reject
REVIEW CRITERIA: PANELS & ALTERNATIVE EVENTS
Quality of Submission (20%)
The proposed topic includes, but is not limited to, one or more of the following criteria:
- exploration of emerging cutting-edge research and design,
- analyses of emerging trends,
- opinions on controversial issues,
- analyses of tools and techniques, or
- contrasting viewpoints from experts in complementary areas of research.
- [includes] an overview of the issues to be discussed.
- Where multiple topics are to be presented, these topics represent a coherent set of related ideas, not a set of separate and distinct papers.
10 - High-quality work. Topic/questions are clearly delineated; the approach is suitable, and the panel approach is well defined. Should create discussion and interaction.
08 - Solid work with a few shortcomings or flaws. Not much offered that is controversial, but good coverage of the subject matter is likely.
06 - Not clear if this is anything new, but coverage of the topic is useful.
04 - Submission has possibilities, but they are unlikely to be realized based on this submission.
02 - Submission does not provide comprehensible structure, not well thought out or conceived.
Clarity of Proposal (15%)
Is the submission well-written? Are the goals, organization, and activities of the panel/alternative event clearly described?
10 - Very well written, a pleasure to read, easy to follow, few or no grammatical or typographical errors.
08 - The essential content will be understood by most readers, but the writing or figures could be improved.
06 - The major points are clear, but the details are either obscured by poor writing or are missing.
04 - Important questions were hard to resolve even; the submission may need editing by an expert.
02 - The submission had major issues with the writing style, which made it difficult to judge the work.
Engagement (25%)
Please assess the interest you expect this panel/alternative event to elicit. Will the proposed topic or activities engage attendees in lively discussions or interactions? Will there be general excitement and buzz about the proposal (i.e., high interest)?
10 - Should be of considerable interest to a large segment of the attendees and/or future readers of the conference proceedings. Should actively engage attendees.
08 - The topic is relatively new or novel, will be of interest to many and is likely to result in active discussion or engagement.
06 - A recurring topic for ASIS&T conferences, likely to be of some continuing interest, with some discussion or engagement.
04 - The topic has been considered previously and lacks on novelty. Unlikely to engage attendees or result in discussions.
02 - Not likely to be of interest to ASIS&T attendees or future readers of the conference Proceedings.
Presenters (20%)
Are the presenters identified? Are they appropriate experts to discuss this topic? Do they represent a sufficiently broad set of viewpoints? Are they the right people for the job (i.e., they will represent the viewpoints well)?
10 - Presenters provide expert knowledge of the topic, and are leaders in the topic area, and diverse. They will generate discussions, thoughts, and interactions.
08 - Presenters provide solid knowledge of the topic and most likely will generate discussions, thoughts, and interactions.
06 - Presenters show moderate knowledge of the topic, not controversial.
04 - Presenters show limited knowledge of the topic, not likely to stimulate discussions, just recitation of what is known already.
02 - Presenters show poor knowledge of the topic, appear to be unaware of recent developments.
Implications for Research, Practice, and/or Society (20%)
Does the panel/alternative event identify implications for research, practice and/or society? For example, does it contribute to our understanding or to a body of knowledge? Or does it discuss how results can be used in teaching, influencing public policy, or improving quality of life? Or does it discuss potential implications for system design and/or practice?
10 - Multiple, significant, and valid implications are identified explicitly and discussed thoroughly.
08 - At least one significant and valid implication is discussed explicitly.
06 - A valid implication is discussed, but it is not a significant implication.
04 - Few or insignificant implications are described.
02 - The proposal does not address the possible implications of the topic.