CS 227  Lecture -*- Outline -*-

focus: exponential complexity of computation tree for recursive version,
	and designing the iteration

* Analyzing the Fibonacci Algorithm (4.6)
	Italian "Son of Bonacci" = filis Bonacci. Mathematician.

-------------
   FIBONACCI NUMBERS

Each pair of rabbits
   has 2 kids.
Mate every month
Bear young 2 months after birth

# of pairs of rabbits:

     1  ; end of month 1
     1  ; end of month 2
     2 = 1 + 1 ; month 3
     3 = 1 + 2 ; month 4
     5 = 2 + 3 ; month 5
     8 = 3 + 5 ; month 6
    13 = 5 + 8 ; month 7
-------------
	Q: How many at the end of month 8?

-------------
	Recursive Definition

(fib 0) = 0
(fib 1) = 1
(fib n) = (+ (fib (- n 1))
	     (fib (- n 2)))
	for n > 1.
-------------
	Note the need for 2 base cases.

** The program
-------------
(define fib ; TYPE: (-> (natural) natural)
  (lambda (int)
    (if (< int 2)
        int
        (+ (fib (- int 1))
	   (fib (- int 2))))))
-------------

	try to compute on-line
		(fib 5)
		(fib 10)
		then (fib 100)
		while that's running... yawn, check your watch,

		then look start to analyze it.

** Analysis
	Trace how this is executed by drawing
	  tree for (fib 4).

	- Point out it is a *binary tree*

	Exercise: Draw tree for (fib 5)
		  using (fib 4) as subtree.

		  Draw tree for (fib 6).

		  How many procedure calls and additions?

	Let (calls-fib n) be the # of procedure calls
	  to compute (fib n).
	Can see by inspection that

-------------
     HOW EXPENSIVE?

(calls-fib 0) = 1
(calls-fib 1) = 1
(calls-fib n)
  = (add1 (+ (calls-fib (- n 1))
	     (calls-fib (- n 2))))
-------------

	Similarly

-------------
(adds-fib 0) = 0
(adds-fib 1) = 0
(adds-fib n)
  = (add1 (+ (adds-fib (- n 1))
	     (adds-fib (- n 2))))
-------------

--------------------
   GRAPH OF TABLE 4.22
140			  a f
130
120
110		      c
100
 90		        a f
 80
 70		    c
 60
 50		      a f
 40		  c
 30	            a f
 20	        c a f
 10    	      c f f
  0 f f	f f f f
        	       	1 1 1 1	1 1
    0 1 2 3 4 5 6 7 8 9 0 1 2 3	4 5
       	       	    n
-------------------
key: f = (fib n), c = (calls-fib n),
     a = (adds-fib n)

	Point out how adds-fib is just about 1 behind calls-fib


** An iterative version of fib
	Need to design a sequence of numbers that will compute it.
	- since need previous 2 fib numbers,
		need to use *two* accumulators.
	Let's play with this for a while, how to start?
		pick the first 2 fib numbers
-------------
    initial values
 for the accumulators

  acc1         acc2
   0            1
   = (fib 0)    = (fib 1)
-------------
	What could we do?  Well, we get the next number by adding them
		but then if we want to do this again, need to save the
		previous one.
	So 1 step of the iteration
		- replace acc1 with old value of acc2 (remains 1 behind)
		- replace acc2 with acc1 + acc2
	have to do this simultaneously.
	     (advantage of recursion over assignment here)

	Emphasize that this sequence has to be designed!!!!
		how?
-------------
acc1   acc2
 0      1
 1      1
 1      2
 2      3
 3      5
 5      8
 8     13
-------------
	- can have 2 students do it (tail recursion)
-------------
(define fib2
  ; TYPE: (-> (natural) natural)
  (lambda (n)
    (if (< n 2)
	n
	(fib-it n 0 1))))

(define fib-it
  ; TYPE: (-> (natural natural natural)
  ;           natural)
  (lambda (int acc1 acc2)
    (if (= int 1)
        acc2
        (fib-it
	  (sub1 int)
	  acc2
	  (+ acc1 acc2)))))
-------------
** How much more efficient is the iterative version?
	Compute (fib 10) (fib 100) (fib 1000)

*** Resources used
	Let (res n) be resources used to compute (fib n)


      RESOURCES USED BY THE VERSIONS

          ITERATIVE    FULLY RECURSIVE
            fib-it       fib
(res n)    (sub1       (add1
             (* 2 n))	(* 3
        		   (sub1
                            (expt
        		      1.618
                              (add1 n))))

(res 10)         19	 595
(res 100)       199    3.8e21
(res 1000)     1999    5.e209
(res 10000)   19999    1e2300

-------------------
     RESOURCES USED BY THE VERSIONS

(res n)

    1e8          r
    1e7          r
    1e6         r
    1e5        r
    1e4	       r                     i
    1e3        r     	     i
    1e2       r      i
    1e1       i
      1 i
        1    1e1    1e2     1e3     1e4
                         n
-------------------
    key: r = fully recursive, i = iterative


  if each resource unit (add or call) takes
     1 nano-second (= 10^{-9}sec) to compute

-------------
               TIME
  if each add/call takes 1 nanosecond

          ITERATIVE    FULLY RECURSIVE
(time 10)     19 ns    0.6 ms
(time 100)    0.2 ms   1e7 years
(time 1000)   2 ms     1e213 years
(time 10000)  20 ms    1e2283 years
-------------
	1e7 = 10^7 = 10,000,000 years
	Universe is only about 5*(10^{9}) years old.

** a warning
	But...

	How much faster is fact using iteration?
	     Not much -- same number of multiplications.
	  Perhaps not enough to justify the example,
		since recursive version of fact is so
		close to definition.

	A warning: it is NOT easy (or desirable)
		   to write iterative procedures for everything.
		    e.g., very hard to get deep-recursion to work iteratively
				(I don't know how)
		   Some procedures have to go through so many gyrations to
			get them to be iterative,
			that they are actually slower!
			(e.g. append)

** terms
-----------------------
    How to talk about efficiency?

               TIME
  if each add/call takes 1 ns
          ITERATIVE    FULLY RECURSIVE
(time 10)     19 ns    0.6 ms
(time 100)    0.2 ms   1e7 years
(time 1000)   2 ms     1e213 years
(time 10000)  20 ms    1e2283 years

  if each add/call takes 0.1 ns
(time 10)     1.9 ns   0.06 ms
(time 100)    .02 ms   1e6 years
(time 1000)   .2 ms    1e212 years
(time 10000)  2.0 ms   1e2282 years
-------------------------
	So there is a qualitative, not just a quantative difference
	in the efficiency; even if we get a computer that is 10 times
	faster, the fully recursive version is still intolerable.

	Constant factors don't matter, cause wiped out by faster machine.

	Hence we summarize in big O notation
---------------
  Big O Notation

          ITERATIVE    FULLY RECURSIVE
            fib-it       fib
(res n)    (sub1       (add1
	     (* 2 n))	(* 3
			   (sub1
                            (expt
			      1.7
                              (add1 n))))

                             (n+1)
             2n-1       3(1.7     -1)+1
                       =          n
                        3(1.7(1.7) -1)+1

                             n
order	     O(n)	O(1.7 )

Def: the resources (res n),
used by an input of size n,
are of order O(f(n)) iff
there is some K such that
 for all sufficently large n,
  (res n) <= K*f(n)
----------------
	The idea is to ignore constant factors and to look at the
	term containing n that makes the biggest difference.

		     2          2              2
		100 n  and 0.5 n   are both O(n )

	But don't ignore constant base of exponent or exponent
		n**2 and n**3 are not the same, neither are 1.7**n and 2**n

                                               n
	So for the recurive fibonacci, is O(1.7 )
	For the iterative, it's O(n)
				    n
	Classes: O(n) is linear, O(c ) for c>1 is exponential
		exponential algorithms are generally impractical as above.

** Using accumulators for list operations

-------------
(define reverse
  ; TYPE: (-> ((list T)) (list T))
  (lambda (ls)
    (reverse-it ls '())))

(define reverse-it
  ; TYPE: (-> ((list T) (list  T))
  ;            (list T))
  (lambda (ls acc)
    (cond
     ((null? ls) acc)
     (else (reverse-it (cdr ls)
		       (cons (car ls)
			     acc))))))
-------------

	Iterative reverse is O(n)
	Recursive version is O(n*n).

** summary
	Q: What is iteration good for?
	Q: What is it not good for?
		So useful, but not always easy or faster

	The point is rather that the design of faster processes (algorithms)
	makes all the difference.  Iteration is just the way we did it
	in this case.