COP 4020 Lecture -*- Outline -*- * Declarative Programming Techniques (3.2-3.5) ** iterative computation (3.2) Why this first? It's the simplest and corresponds to looping. Q: What is an iterative computation? *** general schema (3.2.1, 3.3.3, 3.4.3) ------------------------------------------ ITERATIVE COMPUTATION (3.2) Is this iterative? fun {SumFromTo I J} if I > J then 0 else I + {SumFromTo I+1 J} end end ------------------------------------------ ... no (the strict function call + can't be moved after the recursive call) it has pending computations define a "pending computation" as: a procedure/function call that must be executed after a recursive call. Note: data structure constructions can often be permuted with calls, so don't generally make something non-iterative Not iterative if has pending computations Draw stack of execution if necessary Q: How to make it iterative? To make iterative: design a sequence of state transformations (See section 3.4.3) - introduce a state as an extra argument (or arguments): fun {SumFromTo I J} fun {SumFromToIter I J R} end in {FromToIter I J __} end - Design sequence of transformation of the arguments (including state) e.g., for call {SumFromTo 1 3} I: 1 1 1 1 J: 3 2 1 0 R: 0 -> 3 -> 5 -> 6 - Write code to make that happen Where to get initial value? What's one step? How to stop? fun {SumFromTo I J} fun {SumFromToIter I J R} if I > J then R else {SumFromToIter I J-1 R+J} end end in {FromToIter I J 0} end - Optionally, avoid passing parts that don't change (like I) fun {SumFromTo I J} fun {SumFromToIter J R} if I > J then R else {SumFromToIter J-1 R+J} end end in {SumFromToIter J 0} end For Sqrt we have: declare Sqrt % the block below defines this... local fun {SqrtMaker Tolerance} fun {$ X} {Iterate fun {$ Root} {Abs Root*Root - X} < Tolerance end fun {$ Root} Root end fun {$ Root} (Root + (Root*Root- X))/(2.0*Root) end % Newton's method 1.0} end end in Sqrt = {SqrtMaker StandardTolerance/2.0} end ------------------------------------------ FOR YOU TO DO Make the following iterative: fun {Product Ls} case Ls of E|Es then E*{Product Es} else 1 end end ------------------------------------------ *** When to use Iteration ------------------------------------------ WHEN TO USE ITERATION 0. When need efficiency 1. When the data doesn't maintain "place" 2. When need to return directly to caller ------------------------------------------ Q: what are some examples of this that we have seen? It's not always best to try to do this, E.g. working with binary trees on an exam ** Data-driven recursion (3.4) *** Programming tips Use the DeleteFromNested example (fully recursive) and write examples to explain it ------------------------------------------ TIPS FOR SAVING TIME WHEN DOING RECURSIVE PROBLEMS 1. Write out your own examples if needed a. base case(s) b. related simpler example 2. Write out the type. Assume the inputs match this type. 3. Use an outline that matches the grammar for the input data type. 4. Fill in the outline by generalizing the examples a. What's returned in the base case(s)? b. How do we get the answer from - recursive call's answer, and - other information? 5. Debug bottom up from subexpressions, replacing recursive calls with expected results. 6. One function per nonterminal. Don't mix 2 recursions in a function. ------------------------------------------ See DeleteFromNestedInClass.oz and DeleteFromNestedInClassTest.oz \insert 'DeleteFromNestedInClass.oz' \insert 'TestingNoStop.oz' {StartTesting 'DeleteFromNestedInClassTest.oz $Revision$'} {Test {Delete [c d c f] c} '==' [d f]} {Test {Delete c|d|c|f|nil c} '==' d|f|nil} {Test {Delete d|c|f|nil c} '==' d|f|nil} {Test {Delete c|f|nil c} '==' f|nil} {Test {Delete nil x} '==' nil} {Test {DeleteFromNested nil x} '==' nil} {Test {DeleteFromNested [[a b] [c d c f]] c} '==' [[a b] [d f]]} {Test {DeleteFromNested [[c d c f]] c} '==' [[d f]]} {Test {DeleteFromNested [abara cadabara whoopee hooray]|nil |[whoopee valence valencia cia fbi]|nil whoopee} '==' [abara cadabara hooray]|nil |[valence valencia cia fbi]|nil} {DoneTesting} We'll see examples in a minute *** Type notation (3.4.1) ------------------------------------------ TYPE NOTATION (GRAMMARS) ::= red | blue | green ::= zero | succ( ) > ::= nil | '|' > Equivalently: > ::= nil | '|'(1: 2: >) > ::= leaf | tree( key: value: left: > right: > ) Examples in these grammars: ------------------------------------------ Q: How does the type definition resemble a grammar? *** Natural Numbers ------------------------------------------ RECURSIVE NATURAL NUMBERS ::= zero | succ ( ) Examples: 0 represented by zero 1 represented by succ(zero) 2 represented by succ(succ(zero)) 3 represented by succ(succ(succ(zero))) ------------------------------------------ % representation creation fun {FromInteger I} if I =< 0 then zero else succ({FromInteger I-1}) end end ------------------------------------------ FOLLOWING THE GRAMMAR FOR RECURSIVE FUNCTIONS OVER NATURAL NUMBERS ::= zero | succ ( ) Outline that follows this grammar: fun {F N} case N of zero then ... % basis [] succ (P) then ...{F P}... % inductive case end end ------------------------------------------ Q: In what sense does that look like the grammar? ------------------------------------------ EXAMPLE DEVELOPMENT Write Plus, which adds two What are some examples? What's the type? What's the outline? How do we fill it in? ------------------------------------------ develop these using examples: base case: {Plus zero succ(zero)} == succ(zero) inductive case: want {Plus succ(succ(succ(zero))) succ(zero)} == succ(succ(succ(succ(zero)))) given {ToInteger succ(succ(zero)) succ(zero)} == succ(succ(succ(zero))) how do we get succ(succ(succ(succ(zero)))) from succ(succ(succ(zero)))? generalizing from this example, {Plus succ(P) N2} == succ({Plus P N2}) so get... % Plus: }: > fun {Plus N1 N2} case N1 of zero then N2 [] succ(P) then succ({Plus P N2}) end end Q: How does the structure of the program resemble the type definition? note the recursion occurs where the type def is recursive ------------------------------------------ FOR YOU TO DO Write Mult that multiplies two s. Write Equals that tells if two s are equal, without using Oz's == on arguments. ------------------------------------------ for Equals, show the induction on the first argument version before optimization by simulataneous induction (on pairs). *** Working with lists (3.4.2.6) ------------------------------------------ RECURSION OVER FLAT LISTS > ::= nil | T '|' > Write Add1Each: {Add1Each nil} == nil {Add1Each 1|3|5|nil} == 2|4|6|nil {Add1Each 3|5|nil} == 4|6|nil ------------------------------------------ how do we get [2 4 6] from 1 and [4 6]? now generalize ------------------------------------------ FOR YOU TO DO Write DeleteFirst: > T}: >> such that {DeleteFirst Ls Sought} returns a new list that is like Ls, but without the first occurrence of Sought in Ls (if any). {Test {DeleteFirst nil b} '==' nil } {Test {DeleteFirst a|b|c|b|nil b} '==' a|c|b|nil } ------------------------------------------ Other potential examples if need more: Reverse (can also do iteratively) Map Subst Lookup in association lists *** structure of data determines structure of code **** non-empty lists ------------------------------------------ GENERALIZING HOW TO WRITE RECURSIONS ::= sing(T) | cons(T ) Write: HeadNEL: }: T> TailNEL: }: > Write AppendNEL: }: > such that {Test {AppendNEL sing(3) sing(4)} '==' cons(3 sing(4))} {Test {AppendNEL cons(5 sing(3)) sing(4)} '==' cons(5 cons(3 sing(4)))} {Test {AppendNEL cons(7 cons(5 sing(3))) cons(8 sing(1))} '==' cons(7 cons(5 cons(3 cons(8 sing(1)))))} ------------------------------------------ See AppendNEL.oz AppendNELTest.oz ------------------------------------------ FOR YOU TO DO Using Max: }: > Write MaxNEL: }: T> such that {MaxNEL sing(3)} == 3 {MaxNEL cons(5 sing(3))} == 5 {MaxNEL cons(3 cons(5 sing(3)))} == 5 ------------------------------------------ See MaxNEL.oz **** Multiple nonterminals ------------------------------------------ MULTIPLE NONTERMINALS Recall our last tip: 6. One function per nonterminal. Don't mix 2 recursions in a function. Example: A computer manufacturer (HAL) sells single computers and "grids" ::= single(name: diskSizes: >) | grid(name: elements: >) ::= > ::= > ::= nil | T '|' > Examples: single(name: "xz400" diskSizes: [2 4 8]) grid(name: "small" elements: [single(name: "xz400" diskSizes: [2 4 8])]) grid(name: "maxFlow" elements: [single(name: "xz400" diskSizes: [2 4 8]) single(name: "9000" diskSizes: [8 16]) grid(name: "small" elements: [single(name: "xz400" diskSizes: [2 4 8])])]) ------------------------------------------ ------------------------------------------ EXAMPLE PROBLEM Write "SuperSize", which (a) puts "super-" in front of each name, (b) doubles all disk sizes. Examples: {Test {SuperSize single(name: "xz400" diskSizes: [2 4 8])} '==' single(name: "super-xz400" diskSizes: [4 8 16])} {Test {SuperSize grid(name: "small" elements: [single(name: "xz400" diskSizes: [2 4 8])])} '==' grid(name: "super-small" elements: [single(name: "super-xz400" diskSizes: [4 8 16])])} {Test {SuperSize grid(name: "maxFlow" elements: [single(name: "xz400" diskSizes: [2 4 8]) single(name: "9000" diskSizes: [8 16]) grid(name: "small" elements: [single(name: "xz400" diskSizes: [2 4 8])])])} '==' grid(name: "super-maxFlow" elements: [single(name: "super-xz400" diskSizes: [4 8 16]) single(name: "super-9000" diskSizes: [16 32]) grid(name: "super-small" elements: [single(name: "super-xz400" diskSizes: [4 8 16])])])} What's the type? What's the outline? How do we fill it in? ------------------------------------------ Show first a wrong version like the following (in SuperSizeWrong.oz): % prepend "super-" to the given string % SuperName: }: > fun {SuperName N} {Append "super-" N} end % An incorrect version that doesn't follow the grammar % SuperSize: >: fun {SuperSize Comp} case Comp of single(name: N diskSizes: Disks) then single(name: {SuperName N} diskSizes: {SuperSize Disks}) % Wrong! Type error [] grid(name: N elements: Computers) then grid(name: {Append "super-" N } elements: {SuperSize Computers}) %% Wrong! the following cases cases shouldn't be here [] nil then nil [] H|T then 2*H|{SuperSize T} %% Wrong, never gets to the following case [] C|Cs then {SuperSize C}|{SuperSize Cs} end end What's wrong with that? It gives the run-time type error: %** Expected type: int or float %** uniformly for all arguments %** In statement: {Number.'*' 2 single(diskSizes:[2 4 8] name:[120 122 52 48 48]) _} See why? Corrected version, that uses helpers (DoubleAll and Map) to deal with the lists, so lists aren't passed to SuperSize (in SuperSize.oz): declare % prepend "super-" to the given string % SuperName: }: > fun {SuperName N} {Append "super-" N} end % DoubleAll: >}: > fun {DoubleAll Ls} {Map Ls fun {$ N} 2*N end} end % SuperSize: >: fun {SuperSize Comp} case Comp of single(name: N diskSizes: Disks) then single(name: {SuperName N} diskSizes: {DoubleAll Disks}) [] grid(name: N elements: Computers) then grid(name: {Append "super-" N } elements: {Map Computers SuperSize}) end end **** More programming language like grammars ------------------------------------------ RECURSION OVER GRAMMARS ::= boolLit( ) | intLit( ) | subExp( ) | equalExp( ) | ifExp( ) Write the following Eval: }: > such that {Eval subExp(intLit(5) intLit(4))} == intLit(1) {Eval equalExp(subExp(intLit(5) intLit(4)) intLit(1))} == boolLit(true) ------------------------------------------ Q: What are the base cases? Q: Where should there be a recursion? Q: Examples for each recursive case? Moral: in general can think of all recursions as recursions over grammars *** Difference Lists (3.4.4) **** Basics of Difference Lists A difference list is an "incomplete data structure", in that it usually contains dataflow variables that are undetermined. It's an ephemeral data structure ------------------------------------------ DIFFERENCE LISTS (3.4.4) Grammar: > ::= > # > Idea L1 # L2 represents list L1 minus elements of L2 invariant: L2 is a tail of L1. Example: (1|2|3|X) # X means (1|2|3|nil) Main advantage: Lists of form (a|b|...|X) # X can be appended in constant time Example: To append (1|2|3|X) # X and (4|5|Y) # Y bind X to (4|5|Y) to get (1|2|3|4|5|Y) # Y ------------------------------------------ Think of a difference list as a subtraction problem that hasn't been solved yet Q: Why not just use (1|2|3|X) instead of (1|2|3|X) # X ? because then we couldn't get to the tail in constant time ------------------------------------------ FOR YOU TO DO Write AppendD: }: > Examples: {AppendD (2|3|X)#X (3|5|Y)#Y} == (2|3|3|5|Y)#Y {AppendD X#X (3|5|Y)#Y} == (3|5|Y)#Y ------------------------------------------ Solution Idea: (X-Y) + (Y-Z) = X-Z ... see page 142 fun {AppendD D1 D2} S1#V1 = D1 S2#V2 = D2 in V1=S2 S1#V2 end Conversions fun {ListToDiffList Ls} case Ls of E|Es then Es1#Es2 = {ListToDiffList Es} in (E|Es1)#Es2 else X in X#X end end fun {DiffListToList DL} LS#V = DL in % Can't pattern match against LS unless it's determined % as will suspend if we pattern match against an undetermined variable. if {IsDet LS} then case LS of E|Es then E|{DiffListToList Es#V} else nil end else nil end end Q: What are the limitations of difference lists? Can only unify the tail variable once, so it's ephemeral (see page 149). **** Applications (skip) Skip Flatten (pages 143-145) as it has an ambiguous grammar (T could be a list and then T's structure is ignored...) where diff lists are used to make things faster The Reversal of a list discussion on page 145, for use in program derivation **** Queues and Performance (3.4.5) Q: What's efficiency issue in implementing FIFO queues in the declarative model? How to achieve constant time performance for insert and delete Differences between the strict functional model and declarative model strict functional: can get constant amortized time dataflow extension: can get constant time Q: What's the difference between ephemeral and persistent data? Ephemeral data can only be used as input to one operation, not repeatedly used as input to many operations. Persistent doesn't have this limit, and can be used concurrently Q: How could we get amortized constant time queues? Keep 2 lists, to-remove and added, and move from added list to the to-remove list when the to-remove list is empty, reversing %% From page 147 of CTM declare fun {NewQueue} q(nil nil) end fun {Check Q} case Q of q(nil R) then q({Reverse R} nil) else Q end end fun {Insert Q X} case Q of q(F R) then {Check q(F X|R)} end end fun {Delete Q X} case Q of q(F R) then F1 in F=X|F1 {Check q(F1 R)} end end fun {IsEmpty Q} case Q of q(F _) then F==nil end end Q: How could we get constant time queues with dataflow variables? Use a difference list to add elements %% From page 148 of CTM declare fun {NewQueue} X in q(0 X X) end fun {Insert Q X} case Q of q(N S E) then E1 in E=X|E1 q(N+1 S E1) end end fun {Delete Q X} case Q of q(N S E) then S1 in S=X|S1 q(N-1 S1 E) end end fun {IsEmpty Q} case Q of q(N _ _) then N==0 end end Q: Can we delete elements from a queue that aren't present? Yes, get undetermined variable, so can block lazily... **** Trees (3.4.6-7) Leave this for them to read and homework **** Parsing (3.4.8) Leave this for them to read and homework ** Time and space efficiency (3.5) (skip, put on supplement) This is known as "pragmatics" Q: What's the recommended general approach for calculating resource usage? Translate to kernel, get a set of equations for the time, solve them The equations handle recursive procedures and are themselves recurive, and need to be solved (or approximated). *** Time (3.5.1) ------------------------------------------ EXECUTION TIMES OF KERNEL INSTRUCTIONS S T(S) skip k X = Y k0*max(size(X),size(Y)) X = myrec(f1:X1 k1 + k2*n ... fn:Xn) X = proc k3' + k3*|FV(S) - {X1,...,Xn}| {$ X1 ... Xn} S end S1 S2 T(S1) + T(S2) local X in S end k4 + T(S) if X then S1 k5 + max(T(S1),T(S2)) else S2 end case X of r(f1:X1...fn:Xn) k7 + max(k6*n+T(S1), T(S2)) then S1 else S2 end {F Y1 ... Yn} T_F(size_F( I_F({Y1,...,Yn}))) where I_F is the subset of used arguments and size_F is a measure function and T_F is a function specific to F ------------------------------------------ Q: Why is the time needed for skip constant? Q: Can unification be done constant time? No, as the book notes, in the worst case it depends on the size of data being unified Q: What's the time to do closure formation? If the closure code contains a precomputed environment skeleton giving the offsets in the parent environment. But the implementation does not do this, so it's linear. Why do we need to store the smallest possible environment with a closure? To save space, otherwise could just use whole environment. Q: How can pattern matching in case be constant time? By only allowing the limited patterns in the kernel. *** Memory usage (3.5.2) Q: What needs to be measured for space? high water mark of used memory rate of memory use (proportional to how hard gc has to work) Q: Which is more important? the high water mark, since that determines where program can run ------------------------------------------ MEMORY CONSUMPTION OF KERNEL INSTRUCTIONS S M(S), in words skip 0 X = Y 0 X = V memsize(V) X = proc k1 {$ X1 ... Xn} + n S end + k2*|FV(S) - {X1,...,Xn}| S1 S2 max(M(S1),M(S2)) local X in S end 1 + M(S) if X then S1 max(M(S1),M(S2)) else S2 end case X of max(n+M(S1), M(S2)) r(f1:X1 ... fn:Xn) then S1 else S2 end {F Y1 ... Yn} M_F(size_F( I_F({Y1,...,Yn}))) where I_F is the subset of used arguments and size_F is a measure function and M_F is a function specific to F ------------------------------------------ memsize(i: Int) = let bits = log_2(abs(i)) in if bits < 28 then 0 else ceiling(bits/32) memsize(f: Float) = 2 memsize(p: ListPair)= 2 memsize(t: L(F_1: V1, ..., F_N: VN)) = 1 + N Q: Does the value always need to be completely created in X=V? No, the VM can tell from the value's form what instructions will be needed to unify it. Q: What's the size of a closure? They say k + n, where n is number of free identifiers in the body and k is a constant *** Amortized complexity (3.5.3) Q: What's amortized complexity? the average cost, over all computations Q: What are the techniques used to compute it? budgeting of various sorts (see text) *** Does performance still matter? (3.5.4) Sure, but see text...