Talks & Other Stuff

Jul 2024
Speak at Albany city Summer Youth Employment Program (SYEP):
Is AI going to steal your job before you get there? Separating fact from fiction from hype.
Apr 2024
CS Department Research Seminar:
How to get your paper rejected
Mar 2024
Speak at First Robotics Tech Valley Regional Competition:
Robotics, Computing, and Engineering at University at Albany
May 2023
Invited presentation at Capital One's Testing Summit:
Test Automation and Modern Testing Practices (slides)
April 2023
Keynote presentation at the 16th International Conference on Software Testing, Verification, & Validation
Test Automation: From slow & weak to fast, flaky, & blind to smart & effective (slides)
Mar 2023
Invited talk at Lund University:
Achievements, open problems, and challenges for test automation (slides)
April 2022
AERA Presentation,
Understanding How Elementary Teachers Respond to a Model of Professional Development for Integrating Computer Science Into Instruction (slides)
July 2021
Keynote presentation at the 33rd International Conference on Software Engineering & Knowledge Engineering
Software engineers don't care about models, but they want automated tests (slides)
April 2021
Best paper award at ICST 2021
Efficiently Finding Data Flow Subsumptions, Marcos Lordello Chaim, Kesina Baral, Jeff Offutt, Mario Concilio Neto, and Roberto Araújo
April 2021
Keynote presentation at the A-Most workshop
Nobody cares about models, but everybody wants automation (slides)
Mar 2021
Responsible Authorship—Plagiarism
Mason’s Responsible Conduct of Research Seminar
Oct 2020
Honored at MODELS 2020 with a 10-year most influential paper award
Modeling presentation layers of web applications for testing, with Ye Wu, Journal of Software and Systems Modeling (SoSyM), April 2010.   (slides)
Oct 2020
Keynote presentation at the ISSRE 2020 PhD Symposium
How to Get Your Paper Rejected
May 2020
Awarded the John Toups Presidential Medal for Excellence in Teaching
News article
Mar 2020
Honored with George Mason University’s Alumni Association Faculty Member of the Year 2020
News article
Jan 2020
Presentation at JPMorgan Chase, New York
Current trends in software testing
Nov 2019
Celebrating 30 years of ISSRE
Most influential papers
Sep 2019
We are hiring! Multiple positions in multiple areas.
GMU’s job posting
Nov 2019
A kind and generous tribute from the editor-in-chief of STVR.
Rob Hieron’s editorial
July 2019
Interview about cloud computing with ABC News 7
NOVA and GMU team up to teach cloud computing
May 2019
Interview with the Software Testing AMPlification for the DevOps Team (STAMP) project
Test automation, devops, mutation testing, & security
Apr 2019
A Piazza lecture series podcast on our multi-university, online-asynchronous course: Engaging Small Discussion Based Classes on Piazza
Feb 2019
Received the 2019 Outstanding Faculty Award from the State Council of Higher Education for Virginia
Nomination packet
Old news

Bio & Contact Info

Contact Information

Address:
1214 Western Avenue, Albany, NY 12222
Office:
405 UAB
Phone:
(1) 518-437-4949
Twitter:
0000-0002-8657-2557
Erdos number:
My Erdös number is 3.

    My Mathematics Genealogy
    Google Scholar  

Bio Highlights

Jeff Offutt has published over 200 refereed research papers, has an h-index of 75 (Google Scholar), and has received funding from many government agencies and companies. Offutt is currently co-PI on an NSF project on integrating Computer Science Standards of Learning into K-5 classrooms. He is also leading projects on making smart tests smarter and exploring the ramifications of minimal mutation. Recent projects include the Google-funded SPARC project, which created a new teaching model for CS1 and CS2 to increase scalability, retention, and diversity, while reducing cheating, and the Testing of Critical System Characteristics (TOCSYC) and PILOT projects at University of Skövde, Sweden. His current research interests include software testing, test automation, usable security, software engineering education, analysis and testing of web applications.

Offutt has received several major awards, including the John Toups Presidential Medal for Faculty Excellence in Teaching in 2020, George Mason University’s Faculty Member of the Year in 2020 by GMU’s Alumni Association, the Outstanding Faculty Award from the State Council of Higher Education for Virginia in 2019, and the George Mason University Teaching Excellence Award, Teaching With Technology, in 2013. He leads the MS in Software Engineering program at GMU, teaches Software Engineering courses at all levels and has developed new courses on several Software Engineering subjects, including web engineering, software testing, construction, design, usability, experimentation, and analysis. His textbook, Introduction to Software Testing, edition 2 (co-authored with Paul Ammann), was published by Cambridge University Press in January 2017 and is the leading worldwide textbook in software testing.

Offutt was editor-in-chief of Wiley’s journal of Software Testing, Verification, and Reliability from 2007 through 2019, co-founded the IEEE International Conference on Software Testing, Verification and Validation (ICST) and was the first steering committee chair. He also has served on dozens of conference program committees and been on the editorial boards for the Springer’s Empirical Software Engineering Journal (2006-), the Journal of Software and Systems Modeling (2002-), the Software Quality Journal (2002-), and IEEE Transactions on Software Engineering (2001-2005), is a regular reviewer for NSF and several major research journals, and has been invited to speak throughout the world. He has been involved as designer, builder, and director for many software proof-of-concept research systems, including muJava, coverage analysis web apps, Mothra, Godzilla, CBat, Mistix, Albert, CoupTest, and SpecTest, which have been used by thousands of software engineering researchers and educators. His inventions, including bypass testing and model-based testing, are used by thousands of software companies and embedded in widely used commercial tools such as Selenium. Offutt has made fundamental contributions to several software testing problems, including mutation, specification-based testing, model-based testing, automatic test data generation, object-oriented testing, input space partitioning, and testing of web applications. He has also published on software metrics, software maintenance, and software engineering education.

Education

Employment

  • since August 2023: Professor and Chair of Computer Science, University at Albany.
  • August 1992 to August 2023: Professor of Software Engineering, George Mason University. Assistant Professor 1992-1996, Associate Professor (tenure) 1996, Full Professor 2005.
  • Part-time and Visiting Positions:
    • since 2002: Part-time visiting professor in the Department of Computer Science, University of Skövde, Skövde Sweden.
    • January 2010 to January 2014: Part-time visiting professor in the Department of Computer Science, University of Linköping, Linköping Sweden.
    • January 2000 to August 2007: Part-time Research Scientist with the National Institute of Standards and Technology’s Information Technology Lab.
  • January 1998 to August 1998: Acting Chair of Information and Software Engineering Department, George Mason University.
  • August 1988 to August 1992: Assistant Professor of Computer Science, Clemson University.
  • January 1983 to August 1988: Graduate Research and Teaching Assistant in the Software Engineering Research Center and in the Department of Information and Computer Science, Georgia Institute of Technology

Research

Overview

When I was chosen as editor-in-chief for STVR, I developed its new tag line: “Useful research in building better software.” This is my goal in research. Real engineers are building real software products for real people every day, and my goal as a scientist is to make that easier. I view research through the lens of the literature I grew up reading that excited my imagination to become a scientist. Science fiction tells stories that could be true if certain things happen and certain scientific advances are made. Star Trek fits this definition. Fantasy tells stories that cannot be true in our universe, assuming magic or some other different rules. Lord of the Rings is fantasy. Most research papers published in software engineering conferences and journals are fantasy, proposing ideas or techniques for which there is no path to eventual use in practice. I strive to write papers for which there is a path to use by real software engineers building real software. I strive to write science fiction.

Much of my research has been in software testing, and has resulted in fundamental results in many sub-fields. I have made dozens of results in mutation analysis and testing, collectively resulting in a comprehensive engineering solution that makes mutation practical for industrial use. Many of my results are incorporated into the commercial tool Certess, built by Certitude Inc. and marketed by SpringSoft. I also developed two mutation research tools for widespread use. I helped design and was the principal developer of Mothra, which was used throughout the testing community in the 1990s and 1980s. I led the team that designed and built muJava, which is currently used by hundreds of students and researchers worldwide. With students and colleagues, I also empirically verified the coupling effect, co-invented the schema-based approach for mutation execution, invented techniques to detect equivalant mutants, empirically demonstrated that weak mutation is a viable alternative to strong mutation, developed several processes for how best to apply mutation, invented algorithms for parallelizing mutation, empirically showed that mutation is superior to several other testing techniques, most notably data flow, and developed several classes of mutation operators, including real-time testing, object-oriented connections, web software, XML, and web services. I am credited with the first model-based testing paper (in UML 1999), designing tests from UML statecharts. I also have contributed several results to automatic test data generation, a difficult topic in which most of the problems are generally undecidable, including inventing constraint-based symbolic and dynamic symobolic evaluation algorithms to generate test data to satisfy statement coverage, branch coverage, data flow coverage, and mutation coverage for unit testing; heuristics for recognizing many infeasible test requirements, and the dynamic domain reduction procedure, still one of the strongest algorithms for automatic test data generation. Many of these results were incorporated into the commercial tool Agitator, and Microsoft’s Pex uses a form of dynamic symbolic evaluation. I also have invented new methods and processes for applying input space partitioning (category-partition).

Coverage critera define rules (test requirements) for designing tests. As part of my research, I, with several colleagues, have invented several new coverage criteria, including base choice coverage, prime path coverage, active clause coverage, coupling-based testing, bypass testing, and inheritance & polymorphism coverage. These included analysis and modeling techniques such as atomic section modeling and finite state machine modeling for web applications, the ping-pong alogrithm for test suite reduction, and OO analysis techniques based on the yo-yo graph.

Many of these advances are used by software development companies, and many have shown up in commercial tools, often with different names. They are also taught in hundreds of software engineering courses and used by hundreds of researchers.

Impact Highlights

Academic publication metrics (Google Scholar, as of December 2023):
  • H-Index: 75
  • i10-Index: 175
  • Citation Count: 24,674

Some of my Favorite Publications

Below are some of my favorite publications. My criteria for inclusion are loose ... some papers are “high impact” by Google Scholar, some have high impact by subjective standards, some are landmark papers, and some simply reflect research that I really enjoyed. You can look at them chronologically (most recent paper first) or by categories. A complete list of papers is in my current CV.

2023
Jamie Colwell, Amy Hutchison, Kristie Gutierrez, Jeff Offutt, and Anya Evmenova. Elementary teachers' perceptions of an online professional development for literacy-focused computer science education. Computer Science Education DOI: 10.1080/08993408.2023.2263831.
CS4All
2023
Fabiano Ferrari, Vinicius Durelli, Sten Andler, Jeff Offutt, Mehrdad Saadatmand, Nils Muellner. On Transforming Tests from Models to Code: A Systematic Literature Review. Wiley's Journal of Software Testing, Verification, and Reliability.
Model-based
2023
Marcos Lordello Chaim, Kesina Baral, Jeff Offutt, Mario Concilio Neto, and Roberto Paulo Andrioli de Araujo. On Subsumption Relationships in Data Flow Testing. Wiley's Journal of Software Testing, Verification, and Reliability.
Subsumption eliminates waste
2023
Eileen J. Manchester, Brittany Miller, Amy C. Hutchison, Erdogan Kaya, Kelley S. Regan, Kristie S. Gutierrez, Anya S. Evmenova, Jamie Colwell, and Jeff Offutt. Exploring Teachers' Experiences with the Compose and Code Platform for Teaching CS to All Learners. American Educational Research Association (AERA) Annual Meeting, Technology as an Agent of Change in Teaching and Learning (TACTL) SIG session, Chicago Illinios.
CS4All
2022
Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, Margaret Gross. Evaluating the role of professional development on elementary teachers' knowledge, comfort, and beliefs related to teaching computer science to students with high-incidence disabilities. Journal of Research on Technology in Education.
CS4All
2022
Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, Margaret Gross. Understanding How Elementary Teachers Respond to a Model of Professional Development for Integrating Computer Science Into Instruction. American Educational Research Association (AERA) Annual Meeting, Technology as an Agent of Change in Teaching and Learning (TACTL) SIG session, San Diego CA.
CS4All
2022
Amy Hutchison, Jamie Colwell, Anya Evmenova, Kristie Gutierrez, and Jeff Offutt. The Effects of the Inclusive Computer Science Model of Professional Development on Teachers' Beliefs and Implementation of Computer Science for Students with High-Incidence Disabilities. SITE 2022 - Society for Information Technology & Teacher Education, San Diego, California.
CS4All
2022
Jeff Offutt and Kesina Baral. Designing Divergent Thinking, Creative Problem Solving Exams. Software Engineering Education and Training (ICSE SEET), Pittsburgh Pennsylvania, USA.
CS Education
2021
Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, and Valerie Taylor. Designing a model of computer science professional development for elementary educators in Inclusive settings. Journal of Technology and Teacher Education, 29(2):165-193.
CS4All
2021
Kesina Baral, Jeff Offutt, Paul Ammann, and Rasika Mohod. Practice Makes Better: Quiz Retake Software to Increase Student Learning. 3rd International Workshop on Education through Advanced Software Engineering and Artificial Intelligence (EASEAI).
CS Education
2021
Marcos Lordello Chaim, Mario Concilio Neto, Jeff Offutt. Graph Representation for Data Flow Coverage. 45th IEEE Computers, Software, and Applications Conference (COMPSAC).
2021
Kesina Baral, Jeff Offutt, and Fiza Mulla. Self determination: A comprehensive strategy for making automated tests more effective and efficient. 13th IEEE International Conference on Software Testing, Verification, and Validation (ICST).
Better automation
2021
Marcos Lordello Chaim, Kesina Baral, Jeff Offutt, Mario Concilio Neto, Roberto Araujo. Efficiently Finding Data Flow Subsumptions. 13th IEEE International Conference on Software Testing, Verification, and Validation (ICST) (Best paper award)
Subsumption eliminates waste
2020
Kesina Baral, and Jeff Offutt. An Empirical Analysis of Blind Tests. 12th IEEE International Conference on Software Testing, Verification, and Validation (ICST), Porto, Portugal.
Tests should be smarter
2020
Alessandro Pizzoleto, Fabiano Ferrari, Lucas Dallilo, and Jeff Offutt. SiMut: Exploring Program Similarity to Support the Cost Reduction of Mutation Testing. Sixteenth IEEE Workshop on Mutation Analysis (Mutation 2020), Porto, Portugal.
2020
Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Jeff Offutt, Andrew Vardas-Doane, and Valerie Taylor. Examining Elementary Teachers's Perceptions of Computer Science Professional Development Aimed at Integrating Computational Thinking and Coding in Grades K-5 Instruction. SITE 2020 - The Society for Information Technology & Teacher Education, New Orleans, Louisiana.
CS4All
2019
Alessandro Viola Pizzoleto, Fabiano Cutigi Ferrari, Jeff Offutt, Leo Fernandes, and Marcio Ribeiro. A Systematic Literature Review of Techniques and Metrics to Reduce the Cost of Mutation Testing. Elsevier's Journal of Systems and Software, Volume 157, November 2019.
SLRs are lots of work.
2019
Jeff Offutt, Birgitta Lindström, and Kesina Baral. Teaching an International Distributed Discussion-Based Course. Foundations on Education in Computer Science, Las Vegas, July 2019.
My coolest course.
2018
Yun Guo, Nan Li, Jeff Offutt, and Ami Motro. Exoneration-based Fault Localization for SQL Predicates. Elsevier's Journal of Systems and Software, October, 2018.
Automatic repairs is the future.
2017
Nan Li and Jeff Offutt. Test Oracle Strategies for Model-based Testing. IEEE Transactions on Software Engineering, 43(4):372-395, April 2017.
TSE is always special.
2017
Jeff Offutt, Paul Ammann, Kinga Dobolyi, Chris Kauffman, Jaime Lester, Upsorn Praphamontripong, Huzefa Rangwala, Sanjeev Setia, Pearl Wang, and Liz White. A Novel Self-Paced Model for Teaching CS1 and CS2. Learning at Scale, April 2017, Boston, USA.
SPARC taught us a lot.
2016
Bob Kurtz, Paul Ammann, Jeff Offutt, Marcio E. Delamaro, Mariet Kurtz, and Nida Gökçe. Analyzing the Validity of Selective Mutation with Dominator Mutants. 24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering, November 2016, Seattle Washington, USA.
Mutation can be cost effective.
2016
Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei. Mutation Operators for Testing Android Apps. Elsevier's Information and Software Technology, special issue from the mutation 2015 workshop, 2016.
Mutating androids is so cool.
2014
Jeff Offutt and Chandra Alluri. An Industrial Study of Applying Input Space Partitioning to Test Financial Calculation Engines. Springer's Empirical Software Engineering journal, 19(3):558-581, June 2014.
We were able to not only demonstrate that ISP helped test financial service software very well, but they let us publish openly. Thanks!
2014
Jeff Offutt,Vasileios Papadimitriou, and Upsorn Praphamontripong. A Case Study on Bypass Testing of Web Applications. Springer's Empirical Software Engineering journal, 19(1):69-104, February 2014.
Being able to apply this concept to real, commercial, applications was fun and rewarding.
2014
Paul Ammann, Marcio E. Delamaro, and Jeff Offutt. Establishing Theoretical Minimal Sets of Mutants. 7th IEEE International Conference on Software Testing, Verification and Validation (ICST 2014), March 2014, Cleveland Ohio, USA.
I believe this research has the potential to revolutionize mutation. Time will tell.
2013
Jeff Offutt. Putting the Engineering into Software Engineering Education. IEEE Software, Jan-Feb 2013, 30(1):96-100.
This paper was an ACM "notable" paper for 2013 and sparked a lot of interesting discussion.
2013
Jing Jin, Jeff Offutt, Nan Zheng, Feng Mao, Aaron Koehl, and Haining Wang. Evasive Robots Masquerading as Human Beings on the Web. 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DNS 2013), June 2013, Budapest Hungary.
How often do software engineering guys publish in top security conferences?
2011
Jeff Offutt. A Mutation Carol: Past, Present and Future. Elsevier's Information and Software Technology, special issue from the mutation 2009 workshop, 53(10):1098-1107, October 2011.
This was a fun historial perspective, based on a keynote talk.
2011
Gary Kaminski, Paul Ammann, and Jeff Offutt. Better Predicate Testing. Sixth Workshop on Automation of Software Test (AST 2011), pages 57-63, May 2011, Honolulu, Hawaii, USA.
This is a new way to reduce the cost of mutation--a proof that some mutants subsume others.
2010
Upsorn Praphamontripong and Jeff Offutt. Applying Mutation Testing to Web Applications. Sixth Workshop on Mutation Analysis (Mutation 2010), April 2010, Paris, France.
Mutating the web is fun. Plus it's Upsorn's first paper.
2009
Jeff Offutt and Ye Wu. Modeling Presentation Layers of Web Applications for Testing. Springer's Software and Systems Modeling, 9(2):257-280, April 2010.
This model captures fundamental aspects of web software that are new, powerful, and risky, thus need to be tested. It was hard to get it published because reviewers didn't underestand the web and because we didn't understand how to write the paper. Nevertheless, the TR had 88 citations.
2008
Paul Ammann and Jeff Offutt. Introduction to Software Testing, Cambridge University Press, 2008.
We started this book because we needed a book for our class, and finished it as a service to the community. 562 citations and counting.
2008
Jeff Offutt, Aynur Abdurazik, and Steve Schach. Quantitatively Measuring Object-Oriented Couplings. Springer's Software Quality Journal, 6(4):489-517, December 2008, doi: 10.1007/s11219-008-9051-x.
I was happy to make this field a little more quantitative.
2007
Leonard Gallagher, Jeff Offutt and Anthony Cincotta. Integration Testing of Object-oriented Components Using Finite State Machines. Journal of Software Testing, Verification, and Reliability, Wiley, 17(1):215-266, January 2007.
Len and I worked for years on this paper through 5 painful revisions. It is one of my two "forever papers," deep, valuable, but too complicated for most readers.
2006
Jeff Offutt, Paul Ammann and Lisa (Ling) Liu. Mutation Testing implements Grammar-Based Testing. Second Workshop on Mutation Analysis (Mutation 2006), pages 93-102, November 2006, Raleigh, NC.
This concept paper let us to a new, and much broader, view of mutation testing. This led to the syntax chapter in the testing textbook.
2006
Jing Guan, Jeff Offutt and Paul Ammann. An Industrial Case Study of Structural Testing Applied to Safety-critical Embedded Software. ACM International Symposium on Empirical Software Engineering, ISESE 2006, pages 272-277, September 2006, Rio de Janeiro, Brazil.
We applied CACC (sort of MCDC) to real software and found real faults, including two safety-critical faults. We need more industrial studies like this. Plus, this was my 100th published paper!
2006
Robert Nilsson, Jeff Offutt and Jonas Mellin. Test Case Generation for Testing of Timeliness. Second Workshop on Model Based Testing. pages 102-121, March 2006, Vienna, Austria.
Robert's dissertation was very cool--we applied mutation to test for timeliness.
2005
Mats Grindal, Jeff Offutt and Sten F. Andler. Combination Testing Strategies: A Survey. Journal of Software Testing, Verification and Reliability, Wiley, 15(2):97-133, September 2005.
Not only did this paper directly lead to the ISP chapter in our book, it has 288 citations (as of June 2014).
2005
Yu-Seung Ma, Jeff Offutt and Yong Rae Kwon. MuJava : An Automated Class Mutation System. Journal of Software Testing, Verification and Reliability, Wiley, 15(2)97-133, June 2005.
muJava has been used by hundreds researchers and students, maybe thousands. It is also one of my most cited paper (413 as of June 2014).
2005
Wuzhi Xu, Jeff Offutt and Juan Luo. Testing Web Services by XML Perturbation. IEEE International Symposium on Software Reliability Engineering, pages 257-266, November 2005, Chicago Illinois.
This was one of the first papers to apply mutation to a non-executable language to generate tests.
2004
Jeff Offutt, Ye Wu, Xiaochen Du and Hong Huang. Bypass Testing of Web Applications. IEEE International Symposium on Software Reliability Engineering, pages 187-197, November 2004, Bretagne France.
I am as proud of bypass testing as any invention. The paper has well over 100 citations.
2003
Steve Schach, Bo Jin, David R. Wright, Gillian Z. Heller, and Jeff Offutt. Quality Impacts of Clandestine Common Coupling. Kluwer's Software Quality Journal, 11(3):211-218, July 2003.
Who knew that common (global) coupling could appear after deployment, as the software evolves? Steve and I had a lot of fun with this project and had some fundamental results in software evolution and maintenance.
2003
Paul Ammann, Jeff Offutt and Hong Huang. Coverage Criteria for Logical Expressions. 2003 International Symposium on Software Reliability Engineering (ISSRE '03). pages 99-107, Denver, CO, November 2003.
While working on the book, Paul and I got tired of the ambiguity within MCDC. So we defined a more general structure for logic-based testing, leading to the active clause coverage criteria family defined in this paper.
2002
Steve Schach, Bo Jin, David Wright, Gillian Z. Heller, and Jeff Offutt. Maintainability of the Linux Kernel. IEE Proceedings Journal: Special Issue on Open Source Software Engineering, 149(1):18-23, February 2002.
This was a very controversial paper, as Linux lovers did not like the results: because of the many dependencies, Linux was hard to maintain and getting worse. Nevertheless, the facts are the facts—that's science.
2002
Yu-Seung Ma, Yong-Rae Kwon, and Jeff Offutt. Inter-class mutation operators for Java. IEEE International Symposium on Software Reliability Engineering, pages 352-363, November 2002.
In 2019, this paper was voted one of the most influential papers across the 30 years ISSRE.
2002
Jeff Offutt. Quality Attributes of Web Software Applications. IEEE Software: Special Issue on Software Engineering of Internet Software, 19(2):25-32, March/April 2002.
My first IEEE Software paper. This was a "big picture view" of what's novel about web software and how it affects software engineering. Not surprisingly, it is one of my most cited papers (292 as of June 2014).
2000
Roger T. Alexander and Jeff Offutt. Criteria for Testing Polymorphic Relationships. The Eleventh IEEE International Symposium on Software Reliability Engineering (ISSRE '00), pages 15-23, San Jose, CA, October 2000.
Roger taught me OO and I taught him research. When he showed me the "yo-yo" graph, it scared me senseless. This research has led me to conclude that polymorphism is too hard for most programmers and should be restricted to experts with advanced licenses and training, kind of like fighter jet aircraft pilots.
2000
Jeff Offutt and Roland Untch. Mutation 2000: Uniting the Orthogonal Mutation. 2000: Mutation Testing in the Twentieth and the Twenty First Centuries. pages 45-55, San Jose, CA, October 2000.
I honestly thought this was my last mutation paper. The message was that "we have learned how to make mutation practical, now it's time for a tool builder to come in and commercialize. Here's how." Mark Hampton heard the talk and built Certess for the digital electronics community, so that part worked. But after the talk, a PhD student from KAIST named Yu-Seung Ma came to me with this crazy idea of applying mutation to OO software ... which led to muJava and a lot more mutation papers on my CV. I still haven't forgiven her.
1999
Jeff Offutt, Zhenyi Jin, and Jie Pan. The Dynamic Domain Reduction Procedure for Test Data Generation. Software Practice and Experience, 29(2):167-193, January 1999.
I believe this is my best research. It also may be the most complicated. The paper was rejected four times before a reviewer wrote 10 pages explaining exactly how badly the paper was written. I owe that person a debt for this paper, and for teaching me how to write. Despite not being in a "top" journal, it has lots of citations (168 as of June 2014) and has impacted practice. Agitar's agitator was based on this paper, and although the Pex researchers never cite it, Pex uses many of the ideas that were first presented in the paper.
1999
Jeff Offutt and Aynur Abdurazik. Generating Tests from UML Specifications. Second International Conference on the Unified Modeling Language (UML '99), pages 416-429, Fort Collins, CO, October 1999.
This paper was the first to design tests from UML models and has been credited with starting model-based testing. How cool is that? 460 citations as of June 2014.
1997
Jeff Offutt and Jie Pan. Automatically Detecting Equivalent Mutants and Infeasible Paths. The Journal of Software Testing, Verification, and Reliability, Wiley, 7(3):165-192, September 1997.
Equivalent mutants dramatically increases the cost of mutation testing, but detecting them is a formally undecidable problem. This paper picked a future work suggestion from my dissertation: If solving constraints will result in a test that kills a mutant, then if the constraint system is unsolvable, the mutant must be equivalent. Jie Pan and I developed a set of rules to detect infeasiblilities in the constraint systems and were able to detect well over 40% of the equivalent mutants. We also realized the same approach addressed the "equivalent" problem with path coverage criteria such as statement, branch, and data flow. At the end of the project we thought of using slicing and mentioned it in future work. Several years later, Hierons and Harman applied slicing to the equivalent mutant problem, obviating this paper.
1996
Mary Jean Harrold, Jeff Offutt, and Kanupriya Tewary. An Approach to Fault Modeling and Fault Seeding using the Program Dependence Graph. The Journal of Systems and Software. 36(3):273-296, March 1997.
The PDG combined Mary Jean's analysis knowledge with my mutation knowledge. We had fun, we learned a lot about writing papers, collaborating, and revising papers. I wish Mary Jean and I had written more papers together.
1996
Jeff Offutt, Gregg Rothermel, Christian Zapf, Roland Untch and Ammei Lee. An Experimental Determination of Sufficient Mutation Operators. ACM Transactions on Software Engineering Methodology. 5(2):99-118, April 1996.
1996
Jeff Offutt and Li Li. Algorithmic Analysis of the Impact of Changes to Object-Oriented Software. 1996 International Conference on Software Maintenance, pages 171-184, Monterey, CA, November 1996.
1996
Jeff Offutt and Jane Hayes. A Semantic Model of Program Faults. ISSTA 96, the International Symposium on Software Testing and Analysis, pages 195-200, San Diego, CA, January 1996.
1994
Jeff Offutt and Stephen D. Lee. An Empirical Evaluation of Weak Mutation. IEEE Transactions on Software Engineering, 20(5):337-344, May 1994
1994
Paul Ammann and Jeff Offutt. Using Formal Methods To Derive Test Frames in Category-Partition Testing. Ninth Annual Conference on Computer Assurance (COMPASS 94), pages 69-80, Gaithersburg, Maryland, June 1994.
The formal methods (Z) was not the strength of this paper. It defined the first criteria for combination strategies (category partitioning, or input space partitioning), each choice and base choice. Both are used widely in industry.
1993
Roland Untch, Mary Jean Harrold, and Jeff Offutt. Mutation Analysis Using Program Schemata. 1993 International Symposium on Software Testing and Analysis, pages 139-148, Cambridge, Massachusetts, June 1993.
1992
Jeff Offutt. Investigations of the Software Testing Coupling Effect. ACM Transactions on Software Engineering Methodology, 1(1):3-18, January 1992.
How cool is it to be the first paper in the first issue of a new journal? Not only that, the paper verified an idea that was a basis for mutation, but was doubted by most researchers at the time.
1991
Rich DeMillo and Jeff Offutt. Constraint-Based Automatic Test Data Generation. IEEE Transactions on Software Engineering, 17(9):900-910, September 1991.
The core of my PhD dissertation, and still my most cited paper. A paper that is still cited almost 25 years later had true impact.
1991
K. N. King and Jeff Offutt. A Fortran Language System for Mutation-Based Software Testing. Software Practice and Experience, 21(7):686-718, July 1991.
1989
Jeff Offutt. The Coupling Effect: Fact or Fiction? Third Software Testing, Analysis, Verification Symposium, pages 131-140, Key West, Florida, December 1989.
1988
Rich DeMillo, Dany Guindi, Kim King, Mike M. McCracken, and Jeff Offutt. An Extended Overview of the Mothra Software Testing Environment. Second Workshop on Software Testing, Verification, and Analysis, pages 142-151, Banff, Canada, July 1988.
The first Mothra publication. Mothra changed software testing. Not only was Mothra a research testing tool that was not vaporware, it was distributed to hundreds of other researchers, with source, and is considered a precursor to open source software. Authorship is strictly alphabetical, as were all Mothra papers.
1987
Jeff Offutt and Kim King. A Fortran 77 Interpreter for Mutation Analysis. 1987 ACM SIGPLAN Symposium on Interpreters and Interpretive Techniques, pages 177-188, St. Paul, MN, June 1987.
1985
Jeff Offutt. Hints on Writing Style for Usenet. Monthly electronic posting to the Usenet newsgroup news.announce.newusers.
This was a blog before blogs existed. I took a very good technical writing class as an undergraduate and wrote this in around 1983 to try to help internet posters learn to write. It wasn't peer reviewed, but it's still all over the web, despite being written a decade before the web existed.
  • General Software Testing
    2023
    Marcos Lordello Chaim, Kesina Baral, Jeff Offutt, Mario Concilio Neto, and Roberto Paulo Andrioli de Araujo. On Subsumption Relationships in Data Flow Testing. Wiley's Journal of Software Testing, Verification, and Reliability.
    Subsumption eliminates waste
    2021
    Marcos Lordello Chaim, Mario Concilio Neto, Jeff Offutt. Graph Representation for Data Flow Coverage. 45th IEEE Computers, Software, and Applications Conference (COMPSAC).
    2021
    Marcos Lordello Chaim, Kesina Baral, Jeff Offutt, Mario Concilio Neto, Roberto Araujo. Efficiently Finding Data Flow Subsumptions. 13th IEEE International Conference on Software Testing, Verification, and Validation (ICST) (Best paper award)
    Subsumption eliminates waste
    2014
    Jeff Offutt and Chandra Alluri. An Industrial Study of Applying Input Space Partitioning to Test Financial Calculation Engines. Springer's Empirical Software Engineering journal, 19(3):558-581, June 2014.
    We were able to not only demonstrate that ISP helped test financial service software very well, but they let us publish openly. Thanks!
    2011
    Gary Kaminski, Paul Ammann, and Jeff Offutt. Better Predicate Testing. Sixth Workshop on Automation of Software Test (AST 2011), pages 57-63, May 2011, Honolulu, Hawaii, USA.
    This is a new way to reduce the cost of mutation--a proof that some mutants subsume others.
    2008
    Paul Ammann and Jeff Offutt. Introduction to Software Testing. Cambridge University Press, 2008.
    We started this book because we needed a book for our class, and finished it as a service to the community. 562 citations and counting.
    2006
    Jing Guan, Jeff Offutt and Paul Ammann. An Industrial Case Study of Structural Testing Applied to Safety-critical Embedded Software. ACM International Symposium on Empirical Software Engineering, ISESE 2006, pages 272-277, September 2006, Rio de Janeiro, Brazil.
    We applied CACC (sort of MCDC) to real software and found real faults, including two safety-critical faults. We need more industrial studies like this. Plus, this was my 100th published paper!
    2006
    Robert Nilsson, Jeff Offutt and Jonas Mellin. Test Case Generation for Testing of Timeliness. Second Workshop on Model Based Testing. pages 102-121, March 2006, Vienna, Austria.
    Robert's dissertation was very cool--we applied mutation to test for timeliness.
    2005
    Mats Grindal, Jeff Offutt and Sten F. Andler. Combination Testing Strategies: A Survey. Journal of Software Testing, Verification and Reliability, Wiley, 15(2):97-133, September 2005.
    Not only did this paper directly lead to the ISP chapter in our book, it has 288 citations (as of June 2014).
    2003
    Paul Ammann, Jeff Offutt and Hong Huang. Coverage Criteria for Logical Expressions. 2003 International Symposium on Software Reliability Engineering (ISSRE �03). pages 99-107, Denver, CO, November 2003.
    While working on the book, Paul and I got tired of the ambiguity within MCDC. So we defined a more general structure for logic-based testing, leading to the active clause coverage criteria family defined in this paper.
    1996
    Mary Jean Harrold, Jeff Offutt, and Kanupriya Tewary. An Approach to Fault Modeling and Fault Seeding using the Program Dependence Graph. The Journal of Systems and Software. 36(3):273-296, March 1997.
    The PDG combined Mary Jean's analysis knowledge with my mutation knowledge. We had fun, we learned a lot about writing papers, collaborating, and revising papers. I wish Mary Jean and I had written more papers together.
    1996
    Jeff Offutt and Jane Hayes. A Semantic Model of Program Faults. ISSTA 96, the International Symposium on Software Testing and Analysis, pages 195-200, San Diego, CA, January 1996.
    1994
    Paul Ammann and Jeff Offutt. Using Formal Methods To Derive Test Frames in Category-Partition Testing. Ninth Annual Conference on Computer Assurance (COMPASS 94), pages 69-80, Gaithersburg, Maryland, June 1994.
    The formal methods (Z) was not the strength of this paper. It defined the first criteria for combination strategies (category partitioning, or input space partitioning), each choice and base choice. Both are used widely in industry.
  • Mutation Analysis and Testing
    2020
    Alessandro Pizzoleto, Fabiano Ferrari, Lucas Dallilo, and Jeff Offutt. SiMut: Exploring Program Similarity to Support the Cost Reduction of Mutation Testing. Sixteenth IEEE Workshop on Mutation Analysis (Mutation 2020), Porto, Portugal.
    2019
    Alessandro Viola Pizzoleto, Fabiano Cutigi Ferrari, Jeff Offutt, Leo Fernandes, and Marcio Ribeiro. A Systematic Literature Review of Techniques and Metrics to Reduce the Cost of Mutation Testing. Elsevier's Journal of Systems and Software, to appear 2019.
    SLRs are lots of work.
    2016
    Bob Kurtz, Paul Ammann, Jeff Offutt, Marcio E. Delamaro, Mariet Kurtz, and Nida Gökçe. Analyzing the Validity of Selective Mutation with Dominator Mutants. 24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering, November 2016, Seattle Washington, USA.
    Mutation can be cost effective.
    2016
    Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei. Mutation Operators for Testing Android Apps. Elsevier's Information and Software Technology, special issue from the mutation 2015 workshop, 2016.
    Mutating androids is so cool.
    2014
    Paul Ammann, Marcio E. Delamaro, and Jeff Offutt. Establishing Theoretical Minimal Sets of Mutants. 7th IEEE International Conference on Software Testing, Verification and Validation (ICST 2014), March 2014, Cleveland Ohio, USA.
    I believe this research has the potential to revolutionize mutation. Time will tell.
    2011
    Jeff Offutt. A Mutation Carol: Past, Present and Future. Elsevier's Information and Software Technology, special issue from the mutation 2009 workshop, 53(10):1098-1107, October 2011.
    This was a fun historial perspective, based on a keynote talk.
    2010
    Upsorn Praphamontripong and Jeff Offutt. Applying Mutation Testing to Web Applications. Sixth Workshop on Mutation Analysis (Mutation 2010), April 2010, Paris, France.
    Mutating the web is fun. Plus it's Upsorn's first paper.
    2006
    Jeff Offutt, Paul Ammann and Lisa (Ling) Liu. Mutation Testing implements Grammar-Based Testing. Second Workshop on Mutation Analysis (Mutation 2006), pages 93-102, November 2006, Raleigh, NC.
    This concept paper let us to a new, and much broader, view of mutation testing. This led to the syntax chapter in the testing textbook.
    2005
    Yu-Seung Ma, Jeff Offutt and Yong Rae Kwon. MuJava : An Automated Class Mutation System. Journal of Software Testing, Verification and Reliability, Wiley, 15(2)97-133, June 2005.
    muJava has been used by hundreds researchers and students, maybe thousands. It is also one of my most cited paper (413 as of June 2014).
    2000
    Jeff Offutt and Roland Untch. Mutation 2000: Uniting the Orthogonal Mutation. 2000: Mutation Testing in the Twentieth and the Twenty First Centuries, pages 45-55, San Jose, CA, October 2000.
    I honestly thought this was my last mutation paper. The message was that "we have learned how to make mutation practical, now it's time for a tool builder to come in and commercialize. Here's how." Mark Hampton heard the talk and built Certess for the digital electronics community, so that part worked. But after the talk, a PhD student from KAIST named Yu-Seung Ma came to me with this crazy idea of applying mutation to OO software ... which led to muJava and a lot more mutation papers on my CV. I still haven't forgiven her.
    1997
    Jeff Offutt and Jie Pan. Automatically Detecting Equivalent Mutants and Infeasible Paths. The Journal of Software Testing, Verification, and Reliability, Wiley, 7(3):165-192, September 1997.
    Equivalent mutants dramatically increases the cost of mutation testing, but detecting them is a formally undecidable problem. This paper picked a future work suggestion from my dissertation: If solving constraints will result in a test that kills a mutant, then if the constraint system is unsolvable, the mutant must be equivalent. Jie Pan and I developed a set of rules to detect infeasiblilities in the constraint systems and were able to detect well over 40% of the equivalent mutants. We also realized the same approach addressed the "equivalent" problem with path coverage criteria such as statement, branch, and data flow. At the end of the project we thought of using slicing and mentioned it in future work. Several years later, Hierons and Harman applied slicing to the equivalent mutant problem, obviating this paper.
    1996
    Jeff Offutt, Gregg Rothermel, Christian Zapf, Roland Untch and Ammei Lee. An Experimental Determination of Sufficient Mutation Operators. ACM Transactions on Software Engineering Methodology. 5(2):99-118, April 1996.
    1994
    Jeff Offutt and Stephen D. Lee. An Empirical Evaluation of Weak Mutation. IEEE Transactions on Software Engineering, 20(5):337-344, May 1994
    1993
    Roland Untch, Mary Jean Harrold, and Jeff Offutt. Mutation Analysis Using Program Schemata. 1993 International Symposium on Software Testing and Analysis, pages 139-148, Cambridge, Massachusetts, June 1993.
    1992
    Jeff Offutt. Investigations of the Software Testing Coupling Effect. ACM Transactions on Software Engineering Methodology, 1(1):3-18, January 1992.
    How cool is it to be the first paper in the first issue of a new journal? Not only that, the paper verified an idea that was a basis for mutation, but was doubted by most researchers at the time.
    1991
    K. N. King and Jeff Offutt. A Fortran Language System for Mutation-Based Software Testing. Software Practice and Experience, 21(7):686-718, July 1991.
    1989
    Jeff Offutt. The Coupling Effect: Fact or Fiction? Third Software Testing, Analysis, Verification Symposium, pages 131-140, Key West, Florida, December 1989.
    1988
    Rich DeMillo, Dany Guindi, Kim King, Mike M. McCracken, and Jeff Offutt. An Extended Overview of the Mothra Software Testing Environment. Second Workshop on Software Testing, Verification, and Analysis, pages 142-151, Banff, Canada, July 1988.
    The first Mothra publication. Mothra changed software testing. Not only was Mothra a research testing tool that was not vaporware, it was distributed to hundreds of other researchers, with source, and is considered a precursor to open source software. Authorship is strictly alphabetical, as were all Mothra papers.
    1987
    Jeff Offutt and Kim King. A Fortran 77 Interpreter for Mutation Analysis. 1987 ACM SIGPLAN Symposium on Interpreters and Interpretive Techniques, pages 177-188, St. Paul, MN, June 1987.
  • Model-Based Testing
    2023
    Fabiano Ferrari, Vinicius Durelli, Sten Andler, Jeff Offutt, Mehrdad Saadatmand, Nils Muellner. On Transforming Tests from Models to Code: A Systematic Literature Review. Wiley's Journal of Software Testing, Verification, and Reliability.
    Model-based
    2017
    Nan Li and Jeff Offutt. Test Oracle Strategies for Model-based Testing. IEEE Transactions on Software Engineering, 43(4):372-395, April 2017.
    TSE is always special.
    1999
    Jeff Offutt and Aynur Abdurazik. Generating Tests from UML Specifications. Second International Conference on the Unified Modeling Language (UML 1999), pages 416-429, Fort Collins, CO, October 1999.
    This paper was the first to design tests from UML models and has been credited with starting model-based testing. How cool is that? 460 citations as of June 2014.
  • Automatic Test Data Generation
    1999
    Jeff Offutt, Zhenyi Jin, and Jie Pan. The Dynamic Domain Reduction Procedure for Test Data Generation. Software Practice and Experience, 29(2):167-193, January 1999.
    I believe this is my best research. It also may be the most complicated. The paper was rejected four times before a reviewer wrote 10 pages explaining exactly how badly the paper was written. I owe that person a debt for this paper, and for teaching me how to write. Despite not being in a "top" journal, it has lots of citations (168 as of June 2014) and has impacted practice. Agitar's agitator was based on this paper, and although the Pex researchers never cite it, Pex uses many of the ideas that were first presented in the paper.
    1991
    Rich DeMillo and Jeff Offutt. Constraint-Based Automatic Test Data Generation. IEEE Transactions on Software Engineering, 17(9):900-910, September 1991.
    The core of my PhD dissertation, and still my most cited paper. A paper that is still cited almost 25 years later had true impact.
  • Object-Oriented Analysis & Testing
    2007
    Leonard Gallagher, Jeff Offutt and Anthony Cincotta. Integration Testing of Object-oriented Components Using Finite State Machines. Journal of Software Testing, Verification, and Reliability, Wiley, 17(1):215-266, January 2007.
    Len and I worked for years on this paper through 5 painful revisions. It is one of my two "forever papers;" deep, valuable, but too complicated for most readers.
    2000
    Roger T. Alexander and Jeff Offutt. Criteria for Testing Polymorphic Relationships. The Eleventh IEEE International Symposium on Software Reliability Engineering (ISSRE '00), pages 15-23, San Jose, CA, October 2000.
    Roger taught me OO and I taught him research. When he showed me the "yo-yo" graph, it scared me senseless. This research has led me to conclude that polymorphism is too hard for most programmers and should be restricted to experts with advanced licenses and training, kind of like fighter jet aircraft pilots.
  • Web Software Engineering
    2014
    Jeff Offutt,Vasileios Papadimitriou, and Upsorn Praphamontripong. A Case Study on Bypass Testing of Web Applications. Springer's Empirical Software Engineering journal, 19(1):69-104, February 2014.
    Being able to apply this concept to real, commercial, applications was fun and rewarding.
    2010
    Upsorn Praphamontripong and Jeff Offutt. Applying Mutation Testing to Web Applications. Sixth Workshop on Mutation Analysis (Mutation 2010), April 2010, Paris, France.
    Mutating the web is fun. Plus it's Upsorn's first paper.
    2009
    Jeff Offutt and Ye Wu. Modeling Presentation Layers of Web Applications for Testing. Springer's Software and Systems Modeling, 9(2):257-280, April 2010.
    This model captures fundamental aspects of web software that are new, powerful, and risky, thus need to be tested. It was hard to get it published because reviewers didn't underestand the web and because we didn't understand how to write the paper. Nevertheless, the TR had 88 citations.
    2005
    Wuzhi Xu, Jeff Offutt and Juan Luo. Testing Web Services by XML Perturbation. IEEE International Symposium on Software Reliability Engineering, pages 257-266, November 2005, Chicago Illinois.
    This was one of the first papers to apply mutation to a non-executable language to generate tests.
    2004
    Jeff Offutt, Ye Wu, Xiaochen Du and Hong Huang. Bypass Testing of Web Applications. IEEE International Symposium on Software Reliability Engineering, pages 187-197, November 2004, Bretagne France.
    I am as proud of bypass testing as any invention. The paper has well over 100 citations.
    2002
    Yu-Seung Ma, Yong-Rae Kwon, and Jeff Offutt. Inter-class mutation operators for Java. IEEE International Symposium on Software Reliability Engineering, pages 352-363, November 2002.
    In 2019, this paper was voted one of the most influential papers across the 30 years ISSRE.
    2002
    Jeff Offutt. Quality Attributes of Web Software Applications. IEEE Software: Special Issue on Software Engineering of Internet Software, 19(2):25-32, March/April 2002.
    My first IEEE Software paper. This was a "big picture view" of what's novel about web software and how it affects software engineering. Not surprisingly, it is one of my most cited papers (292 as of June 2014).
  • Other Software Engineering
    2018
    Yun Guo, Nan Li, Jeff Offutt, and Ami Motro. Exoneration-based Fault Localization for SQL Predicates. Elsevier's Journal of Systems and Software, October, 2018.
    Automatic repairs is the future.
    2008
    Jeff Offutt, Aynur Abdurazik, and Steve Schach. Quantitatively Measuring Object-Oriented Couplings. Springer's Software Quality Journal, 6(4):489-517, December 2008, doi: 10.1007/s11219-008-9051-x.
    I was happy to make this field a little more quantitative.
    2003
    Steve Schach, Bo Jin, David R. Wright, Gillian Z. Heller, and Jeff Offutt. Quality Impacts of Clandestine Common Coupling. Kluwer's Software Quality Journal, 11(3):211-218, July 2003.
    Who knew that common (global) coupling could appear after deployment, as the software evolves? Steve and I had a lot of fun with this project and had some fundamental results in software evolution and maintenance.
    2002
    Steve Schach, Bo Jin, David Wright, Gillian Z. Heller, and Jeff Offutt. Maintainability of the Linux Kernel. IEE Proceedings Journal: Special Issue on Open Source Software Engineering, 149(1):18-23, February 2002.
    This was a very controversial paper, as Linux lovers did not like the results: because of the many dependencies, Linux was hard to maintain and getting worse. Nevertheless, the facts are the facts—that's science.
    1996
    Jeff Offutt and Li Li. Algorithmic Analysis of the Impact of Changes to Object-Oriented Software. 1996 International Conference on Software Maintenance, pages 171-184, Monterey, CA, November 1996.
  • Software Engineering and Computer Science Education
    2023
    Jamie Colwell, Amy Hutchison, Kristie Gutierrez, Jeff Offutt, and Anya Evmenova. Elementary teachers' perceptions of an online professional development for literacy-focused computer science education. Computer Science Education DOI: 10.1080/08993408.2023.2263831.
    CS4All
    2023
    Eileen J. Manchester, Brittany Miller, Amy C. Hutchison, Erdogan Kaya, Kelley S. Regan, Kristie S. Gutierrez, Anya S. Evmenova, Jamie Colwell, and Jeff Offutt. Exploring Teachers' Experiences with the Compose and Code Platform for Teaching CS to All Learners. American Educational Research Association (AERA) Annual Meeting, Technology as an Agent of Change in Teaching and Learning (TACTL) SIG session, Chicago Illinios.
    CS4All
    2022
    Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, Margaret Gross. Evaluating the role of professional development on elementary teachers' knowledge, comfort, and beliefs related to teaching computer science to students with high-incidence disabilities. Journal of Research on Technology in Education.
    CS4All
    2022
    Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, Margaret Gross. Understanding How Elementary Teachers Respond to a Model of Professional Development for Integrating Computer Science Into Instruction. American Educational Research Association (AERA) Annual Meeting, Technology as an Agent of Change in Teaching and Learning (TACTL) SIG session, San Diego CA.
    CS4All
    2022
    Amy Hutchison, Jamie Colwell, Anya Evmenova, Kristie Gutierrez, and Jeff Offutt. The Effects of the Inclusive Computer Science Model of Professional Development on Teachers' Beliefs and Implementation of Computer Science for Students with High-Incidence Disabilities. SITE 2022 - Society for Information Technology & Teacher Education, San Diego, California.
    CS4All
    2022
    Jeff Offutt and Kesina Baral. Designing Divergent Thinking, Creative Problem Solving Exams. Software Engineering Education and Training (ICSE SEET), Pittsburgh Pennsylvania, USA.
    CS Education
    2021
    Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Anya Evmenova, Jeff Offutt, and Valerie Taylor. Designing a model of computer science professional development for elementary educators in Inclusive settings. Journal of Technology and Teacher Education, 29(2):165-193.
    CS4All
    2021
    Kesina Baral, Jeff Offutt, Paul Ammann, and Rasika Mohod. Practice Makes Better: Quiz Retake Software to Increase Student Learning. 3rd International Workshop on Education through Advanced Software Engineering and Artificial Intelligence (EASEAI).
    CS Education
    2021
    Kesina Baral, Jeff Offutt, and Fiza Mulla. Self determination: A comprehensive strategy for making automated tests more effective and efficient. 13th IEEE International Conference on Software Testing, Verification, and Validation (ICST).
    Better automation
    2020
    Kesina Baral, and Jeff Offutt. An Empirical Analysis of Blind Tests. 12th IEEE International Conference on Software Testing, Verification, and Validation (ICST), Porto, Portugal.
    Tests should be smarter
    2020
    Amy Hutchison, Jamie Colwell, Kristie Gutierrez, Jeff Offutt, Andrew Vardas-Doane, and Valerie Taylor. Examining Elementary Teachers's Perceptions of Computer Science Professional Development Aimed at Integrating Computational Thinking and Coding in Grades K-5 Instruction. SITE 2020 - The Society for Information Technology & Teacher Education, New Orleans, Louisiana.
    CS4All
    2019
    Jeff Offutt, Birgitta Lindström, and Kesina Baral. Teaching an International Distributed Discussion-Based Course. Foundations on Education in Computer Science, Las Vegas, July 2019.
    My coolest course.
    2017
    Jeff Offutt, Paul Ammann, Kinga Dobolyi, Chris Kauffman, Jaime Lester, Upsorn Praphamontripong, Huzefa Rangwala, Sanjeev Setia, Pearl Wang, and Liz White. A Novel Self-Paced Model for Teaching CS1 and CS2. Learning at Scale, April 2017, Boston, USA.
    SPARC taught us a lot.
    2013
    Jeff Offutt. Putting the Engineering into Software Engineering Education. IEEE Software, Jan-Feb 2013, 30(1):96-100.
    This paper was an ACM "notable" paper for 2013 and sparked a lot of interesting discussion.
  • Software Security
    2013
    Jing Jin, Jeff Offutt, Nan Zheng, Feng Mao, Aaron Koehl, and Haining Wang. Evasive Robots Masquerading as Human Beings on the Web. 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DNS 2013), June 2013, Budapest Hungary.
    How often to software engineering guys publish in top security conferences?
  • Miscellaneous
    1985
    Jeff Offutt. Hints on Writing Style for Usenet. Monthly electronic posting to the Usenet newsgroup news.announce.newusers.
    This was a blog before blogs existed. I took a very good technical writing class as an undergraduate and wrote this in around 1983 to try to help internet posters learn to write. It wasn't peer reviewed, but it's still all over the web, despite being written a decade before the web existed.
Jan 2024 COMMENTING THIS SECTION OUT--Maybe I'll bring it back if I ever decide to update it.

Keynote Talks & Other Public Lectures

I have completely lost track of all the talks I have given. This list is just a few of the talks that I remember.
2014
How to Get Your Paper Rejected. Invited talk for the PhD Symposium at the International Conference on Software Testing, Verification and Validation (March 2014) PPTX
Generating Tests from Behavioral Models. Invited talk at Siemens Research Labs, Princeton NJ (May 2014) PPTX
2013
How the Web Brought Evolution Back Into Design. Keynote Speaker at the Brazilian Symposium on Software Engineering (SBES), Brazilia Brazil (October 2013) PPTX
Research Issues in Web Application Testing. Invited Lecturer for the Intl Summer School on Trends in Computing, Tarragona Spain (July 2013)
Slides: Part 1 Part 2 Part 3 Part 4
Invited lectured to the INFORTE Summer School, Oulu Finland (June 2013)
2012
Better Logic-Based Testing. Invited talk at Linköping University (January 2012) PPTX
Using Collaborative Learning and Divergent Thinking to Teach Software Engineering. Keynote talk at the Software Engineering Education Conference, Kunpur India (February 2012) PPTX
Teaching an Innovative Asynchronous, International, Multi-University Seminar. Invited talk at the Innovations in Teaching & Learning Conference George Mason University (September 2012) PPTX
2011
Testing Web Applications with the Atomic Section Model. Invited talk at Linköping University (January 2011) PPTX
Cost / Benefit Arguments for Automation and Coverage. Invited talk to the Northern Virginia Test Automation Interest Group (August 2011) PPTX
2010
Automatically Generating Test Data for Web Applications. Keynote speaker at the Google Test Automation Conference (GTAC), Hyderabad India (October 2010) PPTX
The Model-Driven Test Design Process. Keynote speaker at SOFTEC, Kuala Lumpur Malaysia (July 2010) PPTX
Testing Dynamic Aspects of Web Applications. Research tutorial at SOFTEC, Kuala Lumpur Malaysia (July 2010) PPTX
The Model-Driven Test Design Process. Seminar lecture at TAROT, Austria (June 2010) PPTX
Testing Dynamic Aspects of Web Applications. Research lecture at TAROT, Austria (June 2010) PPTX
2009
Automatic Test Data Generation : Who, When and Where?. Invited talk at University of Boston (December 2009) PPTX
The Model-Driven Test Design Process. Invited talk at Telechips, Seoul Korea (October 2009) PPTX
Introduction to Software Testing. Two day seminar at Samsung Electronics, Suwon Korea (October 2009)
Introduction to Software Testing. Full-day seminar at Sogang University, Seoul Korea (October 2009)
Model-Driven Test Design. Keynote speaker at the 12th International Conference on Quality Engineering in Software Technology (Conquest), Nuremberg Germany (September 2009) PPTX
Automatic Test Data Generation: Who, When and Where?. Keynote speaker at the third IEEE International Conference on Secure Software Integration and Reliability Improvement, Shanghai China (October 2009) PPTX
A Mutation Carol: Past, Present and Future. Keynote speaker at the Mutation 2009 workshop, Denver CO (April 2009)
2008
XML-based Testing of Web Software Services. Keynote speaker at the 2nd International Research Workshop on Advances and Innovations in Systems Testing, Memphis TN (May 2008) PPTX
Programmers Ain't Mathematicians and Neither Are Testers. Keynote speaker at the 10th International Conference on Formal Engineering Methods, Japan (September 2008) PPTX
2007
Internet Safety. Invited speaker at Wakefield School in The Plains, VA (October 2007)
2006
How to be a software testing PhD student. Invited Lecturer for the ARTES Summer School (A network for Real-Time research and graduate Education in Sweden), Nasslingen, Sweden (August 2006)
ABOVE COMMENTED OUT JANUARY 2024

Honors and Awards

  • 2021 ISSRE 30-year Most Influential Paper award (Inter-class mutation operators for Java, 2002)
  • 2021 ICST Best paper award (Efficiently Finding Data Flow Subsumptions)
  • 2020 SoSyM / MODELS 10-year Most Influential Paper award (Modeling presentation layers of web applications for testing, 2010)
  • 2020 John Toups Presidential Medal for Faculty Excellence in Teaching (George Mason University)
  • 2020 Faculty of the Year award (GMU’s alumni association)
  • 2019 Outstanding Faculty Award (State Council of Higher Education for Virginia)
  • 2018 Mutation Workshop Best paper award (A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results)
  • 2013 Notable article for "Putting the Engineering into Software Engineering Education," IEEE Software 30(1), February 2013. Selected by ACM Computing Reviews
  • 2013 George Mason University Teaching Excellence Award (Teaching With Technology)
  • 2012 Finalist, Governor's Technology in Education Award (Commonwealth of Virginia)
  • 2010 & 2009 GMU SCHEV Outstanding Faculty member (George Mason University)
  • 2006 Outstanding Researcher Award (ISE Department)
  • 2004 Outstanding Researcher Award (School of IT&E)
  • 2003 Best Teacher Award (School of IT&E)
  • 2003 Outstanding Teacher Award (ISE Department)
  • 1996 ICECCS Outstanding paper award (Coupling-based Integration Testing)

Teaching & Mentoring

Courses

I have taught for 35 years at four universities, University at Albany, George Mason University, Clemson University, and the Georgia Institute of Technology. Most of my courses are in Software Engineering, although I taught data structures, introductory programming, compilers, and algorithms early in my career.

I have introduced a large number of innovations in terms of topics, teaching techniques, and style. At Mason, this includes developing six completely new courses, substantially revising six others, and publishing five education related papers. I also pioneered several innovations for incorporating the web into class material delivery, some of which are now used by many other college professors. My course designs for INFS 590, SWE 637, SWE 619, SWE 632, SWE 642, and SWE 763 are used by other teachers at Mason and elsewhere.

Some general information about my courses. Courses that I have created from scratch are marked with a light bulb (💡) and courses that I refactored are marked with a gapped circle arrow (⟳).

Courses at U @ Albany Graduate Courses at GMU Undergraduate Courses at GMU

Philosophy

Professors in engineering departments at research schools like Mason have an odd, sometimes contradictory relationship with teaching. I was hired for my research and tenured for research. I am recognized in my department for prolific publishing. Where does teaching fit in? To teach well, we have to enjoy teaching and care about the students. If we don't care for them, they will not care what we say. I like to teach because I am addicted to learning. I get bored if I don't learn something new every day, so I teach because it gives me an excuse to learn. This compulsion drives me to teach new subjects and to find new, innovative ways to present knowledge.

I started teaching with chalk boards, moved to white boards with colored markers, then plastic transparencies, and on to power point. I first used the web to disseminate class materials and display examples, then broadcast my lectures to synchronous, simultaneous audiences outside the classroom. I then found I could use the web to extend learning beyond the temporal and spatial constraints of the classroom through interesting and informative discussions, using recorded lectures to run a “flipped” classroom, teaching asynchronously, completely on the web.

When I started teaching as a 24 year old graduate student, I had a very peer-like relationship, then became more of a big brother, and as I grew older, developed more of a father-figure approach. Through all of this, one of the most enjoyable parts of teaching has been enduring relationships with people I genuinely like and respect.

I grew up in an educational desert. None of my friends’ parents went to college, my elementary school classes were taught to the lowest common denominators, classes were tortuously slow, and I was a constant target of bullies. 40% of my high school class dropped out and 10 of 150 started college. My geometry teacher couldn't do proofs, my chemistry class had no chemicals or even running water, and my 12th grade English class included functionally illiterate students. I graduated high school poorly educated, but hungry. Literally starved for knowledge.

No matter how much I learn, that hunger has never gone away. My hunger for knowledge has never abated and it has become a passion for teaching. My goal is to impart that passion for learning to my students. I love to be the teacher I seldom had.

My classes emphasize diversity, divergent thinking, and collaborative learning.
My teaching principles.
Students who want to email me should read my suggestions for students making requests first.

Mentoring

Current PhD Students
  • David Farmer. Conformance testing of frameworks
  • Anders Eriksson. Skövde University, co-advising with Birgitta Lindström. Model-based testing.
  • Andras Markl. Skövde University, co-advising with Birgitta Lindström. Test criteria.
Some relevant advice ... part-time or full-time?
PhD Graduates Masters Students
  • Chandra Alluri. Thesis title: Testing Calculation Engines Using Input Space Partitioning and Automation. (PDF of Thesis) (PDF of Presentation) (2008)
  • Vasileios Papadimitriou. Thesis title: Automating Bypass Testing for Web Applications. (PDF of Thesis) (PPT presentation) (2006)
  • Wei Ding, Using Mutation to Generate Tests from Specifications (thesis advisor: Paul Ammann), 2000, Professor of Computer Science, University of Massachusetts Boston. Wei and her professors
  • Aynur Abdurazik, Generating Test Data From UML Specifications, 1999.
  • Alisa Irvine, The Effectiveness of Category-partition Testing of Object-oriented Software, (1994).
  • Jie Pan, Using Constraints to Detect Equivalent Mutants (1994).
  • Christian Zapf, Distributing Mutation on a Network of Sun Servers (1993). (PDF of Thesis)
  • Tracey Oakes, A WIMP Interface to Mothra (1993).
  • David Pressley, Data Flow Analysis for Generating Statement Coverage Constraints (1992).
  • Raad Yacu, An Improved Procedure for Generating Statement Coverage Constraints, 1991.
  • Scott Fichter, Parallelizing Mutation on a Hypercube (1991).
  • Stephen D. Lee, Weak vs. Strong: An Empirical Comparison of Mutation Variants (1991).
  • W. Michael Craft, Detecting Equivalent Mutants Using Compiler Optimization Techniques (1989).
  • Jason Emil Seaman, Using Symbolic Evaluation to Address the Internal Variable Problem (1989).

Curriculum Development

  • Co-led team to create an undergraduate concentration in Software Engineering as part of the Applied Computing Science degree (effective Fall 2009)
  • Led team to create an undergraduate minor in Software Engineering (effective Fall 2006)
  • Led team to a major restructure of GMU’s MS program in Software Engineering (effective Fall 2005)
  • Developed a graduate Certificate in Web Software Engineering
  • Led team to develop a PhD Concentration in Software Engineering within GMU’s PhD in Information Technology
  • With Henry Hamburger of GMU’s CS Department, led team to design GMU’s PhD in Computer Science (effective Fall 2000)
  • Participated in major restructuring of Clemson University’s MS in Computer Science, eliminating requirement for MS project (wrote first draft of document)
  • Participated in major restructuring of Clemson University’s PhD in Computer Science (wrote first draft of document)

Software

Throughout my career I have focused on software engineering techniques that incorporate a large theoretical component, and also that can be used by real programmers. I emphasize empirical validations, and as such, have constructed, helped to construct, or led the construction of several working software systems. Most of these offer a proof-of-concept demonstration vehicle of theoretical concepts as well as a lab for empirical validation. Some of these (most notably muJava, Mothra and Godzilla, and also Mistix and Albert) have been used by dozens or hundreds of other researchers.

muJava

The mutation testing for Java project (μJava) is a testing tool that is distributed under an open source software model. μJava is a mutation testing system for Java programs that supports the object-oriented features of inheritance, polymorphism and dynamic binding. It was built as an international collaborative effort between myself at GMU and Yu-Seung Ma at the Korean Advanced Institute for Science and Technology (KAIST), as part of her PhD work. μJava is freely available on its website. and is currently being used at hundreds of universities for research and teaching purposes. Three papers have been published about μJava, it was featured at the research tool demo session at the International Conference on Software Engineering in 2006, and many dozen of researchers and educators use μJava.

Coverage-based Analysis Tool (CBAT)

This NSF-funded research project created a fully functional, multi-capability, robust analysis and coverage tool for Java. This tool allows researchers to generate a large number of coverage graphs that represent Java programs, which can then be used for test generation, test coverage measurement, metrics computation, maintenance computations, and other purposes. CBAT focused on inheritance and polymorphic relationships, as well as traditional control and data flow information. This tool was built as part of Roger Alexander’s PhD dissertation.

Coupling-based Testing (CoupTest)

This NSF-funded research project created an analysis tool to extract coupling relationships and measure coverage of tests according to the coupling-based test technique. This tool was built in Java for Java programs.

Specification-based Testing (SpecTest)

This Rockwell/Collins-funded research project developed a tool to measure the extent to which system-level test data satisfies a set of test criteria that are defined on formal specifications and design models of the software. The tool worked with the NRL’s SCRTool, which implements the SCR specification language, and Rational Corporation’s Rose tool, which supports the Unified Modeling Language. This tool was primarily built by Aynur Abdurazik as part of her MS thesis.

Program Mutation for SDI Applications (Mothra)

This research project involved approximately 12 people in the design and development of a mutation-based unit-level software testing system known as Mothra. Mothra was over 50,000 lines of C code, executed in Unix environments, and among other tools includes a parser, an interpreter, a test harness, a test case manager, and several user interfaces. While a graduate student, Dr. Offutt and Dr. Rich DeMillo (currently of Georgia Institute of Technology) were the major designers of Mothra, and Offutt was responsible for over 50% of the implementation, including the interpreter, the test harness, the test case manager, and the initial user interfaces. This work yielded many dozens of papers in refereed conferences and journals; Mothra was been used as a research and demonstration vehicle in dozens of locations.

Godzilla

This tool was designed and built as part of Offutt’s doctoral work. Godzilla automatically generated test data for unit testing according to several test criteria, including mutation, multiple condition coverage, branch and statement. Godzilla was approximately 25,000 lines of C code, executed in Unix environments, and was fully integrated with Mothra. The software company Agitar incorporated much of the Godzilla’s test data generation innovations into their commercial testing tool, Agitator.

HyperMothra, Leonardo, and Equalizer

These proof-of-concept systems all involve major modifications to Mothra to improve the efficiency of the mutation testing process. These systems provide extended functionality, use different basic algorithms, and utilize parallel hardware architectures. These projects were all implemented by graduate students as Master’s projects, using Offutt’s design and under his direction.

IMSCU

This streamlined mutation system was built by several graduate students according to Offutt’s specification and under his direction. Two versions were built, one in C (about 8,000 lines of code) and another in Modula-2 (about 5,000 lines). IMSCU was used in several course projects at both the graduate and undergraduate level, and as a research vehicle in software metrics experimentation. In an undergraduate senior-level project-oriented course, IMSCU was used as a project template, where the class was supplied with a partial implementation and a system architectural design, and divided into five teams, each of which derived requirements for, designed, implemented, and tested an additional major subsystem. All subsystems were integrated into a complete system, which was then tested as a whole. This project involved reuse, maintenance, integration, and all phases of the software life cycle, and was managed by Offutt to provide practical, industrial-oriented experience to the students.

Mistix

This simplified file system was initially specified by Dr. Offutt for a class implementation project. It has been used in several classes for implementation projects, and has been implemented in several languages (C, Modula-2, Ada) by Offutt. Implementations have been used in test classes, user interface classes, and the concept has been used in formal methods classes. Implementations have also been used in several research projects.

Albert

This project to build a computer algebra system for nonassociative identities was led by Dr. Jacobs of Clemson University. Offutt designed both the overall system and a command-language user interface, which were implemented by graduate students. Albert is currently used by mathematicians around the world to support research in nonassociative algebra.

Leadership, Service & Outreach

Highlights

Offutt has been a leader at various levels. Since 2007, he has been editor-in-chief of the journal of Software Testing, Verification, and Reliability. In 2008 he and a small team (Andrews, Briand, and Baudry) created the International Conference on Software Testing, Verification and Validation (ICST), which immediately became one of the largest and most successful area conferences in software engineering. He was elected the first Steering Committee Chair and was co-Program Chair for ICST 2009. His textbook, Introduction to Software Testing (co-authored with Ammann), is the leading worldwide textbook in software testing.

Software Engineering

Offutt also is or has been on the editorial board for several journals, including TSE, EmSE, SQJ, and SoSyM. He has also helped organize over a dozen workshops and PhD Symposiums, and been on the technical program committees for dozens of conferences. Over his 26 year career, he has reviewed hundreds of papers and proposals.

At George Mason

From 2005 through 2013, and starting again in 2015, Offutt has led George Mason’s MS program in Software Engineering. He has chaired several departmental and university committees, including a Provost’s task force for ethical computing, graduate studies committees, tenure and promotion committees, several hiring committees, and been assistant department chair and department chair (interim). He has also served on dozens of other committees at all levels at the university.

Community Outreach

Outside of academic life, Offutt is a regular speaker at public schools and local civic organizations on topics such as internet safety, web services, software quality, and empirical research. He is also a certified little league baseball umpire, and has coached youth teams in basketball, baseball, and Odyssey of the Mind.

Consulting Activities

Most of my consulting is with teaching & training, or with software IP issues as expert analyst and expert witness.

  • Various Software Intellectual Property Cases
  • National Wooden Pallet and Container Association
  • Samsung Electronics
  • Certess Inc, member of the Technical Advisory Board
  • Creative Labs
  • Agitar Software
  • Hyperchip, Inc.
  • IP Optical, Inc.
  • United Arab Emirates University
  • Bell Communications Research
  • INternational Research Institute
  • Reliable Software Technologies
  • Rockwell Collins Avionics

Personal

This sections contains things that have little or nothing to do with my professional life, but that I can share.