I. Discussion about Reports What is science? What is a scientific claim? A. judging tools and papers 1. industry ------------------------------------------ HOW TOOLS/PAPERS ARE JUDGED IN INDUSTRY What do managers/developers look for? ------------------------------------------ 2. academia ------------------------------------------ HOW PAPERS ARE JUDGED IN ACADEMIA Readers/reviewers look for: ------------------------------------------ What does "generality" mean in Computing? Why is generality important? B. abstract ------------------------------------------ WRITING AN ABSTRACT Why? Model 1: Summary of paper - Context - Problem - Approach - Claims and Results - Benefits Model 2 (4 sentences by Ken Beck): ------------------------------------------ C. attack models ------------------------------------------ WHAT IS AN ATTACK MODEL? Describe: - Assumed capabilites of attacker - What attacker can do, during an attack NOT: - How attack proceeds - What attacker wants to do after attack ------------------------------------------ Why do we want to assume attackers have certain capabilities? What is the name for what an attacker wants to do after an attack succeeds? ------------------------------------------ WHY SPECIFY AN ATTACK MODEL? So reader can judge: ------------------------------------------ D. describing the approach E. related work ------------------------------------------ FINDING RELATED WORK Resources: scholar.google.com portal.acm.org ieeexplore.ieee.org springerlink.com usenix.org/publications/ library.ucf.edu ------------------------------------------ The resources are good for finding academic works, but how would you find commercial products? ------------------------------------------ WHAT TO LOOK FOR IN RELATED WORKS What problem is being solved? Is it the same problem? What are the differences? Is it available for use? What claims are made for the solution? Is that better than our solution? What are the differences? What are the pros and cons? What approach is used in their solution? Are there any good ideas we can use? How hard was it to implement? ------------------------------------------ F. evaluation ------------------------------------------ EVALUATION: DO MORE THAN ONE! If an engineer says that a tool works in one case, would you use it in your company? ------------------------------------------ G. example 1. topic ------------------------------------------ EXAMPLE TOPIC Problem: preventing integer overflows Wrap-around can cause: - allocation of 0 bytes - logic errors Complications: - implicit coercions e.g., unsigned int to int - sign extension - pointer arithmetic uses different types in C (size_t vs. ptrdiff_t) 4 CWEs: 682: incorrect calculation 190: integer overflow or wraparound 191: integer underflow 192: integer coercion error ------------------------------------------ 2. Approach ------------------------------------------ WHAT APPROACH TO USE? Basic decision: static vs. dynamic What are the pros and cons? So what should be the plan? ------------------------------------------ Which will have a smaller trusted computing base? 3. title ------------------------------------------ TITLE Ideas: - mention key words - describe problem ("Preventing, ...") - include any limiting context - describe kind of solution - fit in claim if possible - use a colon for a subtitle Example: ------------------------------------------ 4. Related Work a. related tools (industry) ------------------------------------------ SEARCHING FOR INDUSTRIAL RELATED WORK Tips: - search for the problem on google - look into compilers (for related languages) ------------------------------------------ b. related papers (academia) ------------------------------------------ SEARCHING FOR ACADMIC RELATED WORK Tips: - search for the problem in scholarly engines - search in multiple places ------------------------------------------ Should we ignore related work that uses a different approach? 5. Describing the Approach (for tool builders) ------------------------------------------ DESCRIBING THE APPROACH Key questions: - what would a CS grad student need to know to work on this? Describe: - overall approach in technical term e.g., static or dynamic analysis - key decisions - user interface or API - modules/components of software use of existing tools - architecture, how components connect - key data structures and algorithms ------------------------------------------ ------------------------------------------ EXAMPLE ------------------------------------------ 6. Evaluation ------------------------------------------ EVALUATION For tool comparisons: case studies showing: - download (availability) - install difficulty - ease of use (for extremes) experiments showing: - effectiveness (for real problems) - amount of imprecision - cost For tools: experiments showing: - effectiveness - amount of imprecision (false positives, false negatives) - (performance) case studies: - showing utility - helping explain ideas ------------------------------------------ ------------------------------------------ DESIGNING EXPERIMENTS Planning: - What are the possible outcomes? - What will those tell us? Need: ------------------------------------------ 7. Related Work ------------------------------------------ RELATED WORK What solves the same problem? Lump any of it together? What advantages/disadvantages vs. related? ------------------------------------------