Cookie Rating Activity
                                      Tom Moore
                                  October 12, 1994

          I began the activity with the following instructions:

                           _______ _____

               In her book Tainted Truth (Simon and Schuster, 1994)
               Cynthia Crossen quotes Robert Pitofsky (former head of
               the Federal Trade Commission's Bureau of Consumer
               Products):  "In markets where product claims are viewed
               with utter suspicion high price is adopted as an
               indication of quality, and price competition and
               product improvement become economically irrational."

          Crossen's point is that with many products we are bombarded with
          so many claims of high quality, often bolstered by questionable
          "studies", that Pitofsky's principle kicks in.

          I propose the following class project to occupy our attention for
          a few class periods.  Let's devise an experiment to see if for
          commercially available chocolate chip cookies price translates
          into quality.  Devise an experiment to answer this question.  By
          tomorrow come up with a plan for the experiment.  I encourage you
          to brainstorm in small groups.  If this is not feasible then work
          on your own.  Generate your ideas for a good experiment with as
          much detail as possible.

          The next day the class came prepared to share their ideas.  We
          spent the hour brainstorming and then sharpening our opinions of
          various ideas.  The main issues the class came up with were:  (1)
          Population:  are we interested in all chocolate chip cookies on
          the market including the cookie dough you bake yourself, soft and
          hard cookies?  (2) Sample and sample size:  How many varieties
          and which ones do we choose?  (3) Details:  How blind do we need
          to make the test.  Do the subjects need to be literally
          blind-folded or can the appearance of the cookie be part of what
          they are reacting to?  Does the researcher need to be blind to
          which cookies are which?  (4) Measurement:  Do we make subjects
          rank cookies or rate them on a 5 or 7 point scale?  What is/are
          the independent variable(s)?  Taste, texture, appearance, overall
          rating?  What is the independent variable?  Price per ounce or
          per cookie?

          We spent considerable time arguing about these points.  By the
          end of class we were decided on most points and I said I would
          make some decisions on the points still in contention and that
          practical considerations would dictate details in some cases.

The next day was test day. I had purchased these cookies for the experiment: Cookie type price/size cost/ounce overall class rating Pepperidge Farm $1.89/5 oz $.378/oz 2.79 Hy Vee Deli 2.49/9 oz .277 3.61 Archway 1.89/11 oz .172 2.29 Keeblers Deluxe 2.99/18 oz .166 3.29 Chips Ahoy 2.99/18 oz .166 2.75 Chip-a-riffic 2.49/18 oz .138 2.29 Shurfine 1.99/16 oz .124 2.71 Hy-Vee generic .99/12 oz .083 2.21 I purchased from two local markets. I used a deli cookie from Hy-Vee. Chip-a-riffic are packaged to look a lot like Chips Ahoy or the Keebler Deluxe. Shurfine is a store brand (Food 4 Less store) as is the Hy-Vee generic brand. The experiment was run this way. I randomly assigned each of the 8 cookies to a brown bag with letters A - H. Before class I put cookie pieces of about the same size (halves or thirds depending on the size of the cookie) in the bags in sufficient quantities. One student, who didn't want to eat cookies, assisted me in the classroom work. We passed out to each student a paper plate that the students divided into geometric sectors with letters A, ..., H. Then we passed bags up and down rows and everyone put a cookie shard of each letter on their plate in the proper sector. I imparted on them the importance of not colluding with their neighbors and essentially being quiet and taking it seriously and they did very well at this. I gave them each a glass of water. I gave each of the 8 sectors of the room (4 in each, there were exactly 32 students tasting) a different order to taste in -- just ABCDEFGH, cycled around 8 times -- to avoid any order effect. They were then allowed to go back and retaste any cookies as often as they liked. They were given the rating sheet below. We had decided as a class to rate on the different factors, and I chose a 1-5 scale (as in "Consumer Reports). I entered the data into Minitab. We spent class time analyzing results. Main findings: There is a modest correlation between quality and price for our sample: R = .49 for the ecological correlation of the 8 points above, R = .2 if you correlate at the level of individual subjects. We also found strong correlations between overall ratings and the other factors: for taste the correlation was .87, for texture .79, and for appearance .62. These are non-ecological correlations. We then spent some time on another day discussing what we'd learned and critiquing the design. Main conclusion: yes, there is a modest correlation. Design criticisms: Maybe don't need all factors since they correlate well with overall rating, giving "pieces" instead of whole cookies makes it hard to rate appearance, I should've stuck with hard cookies for the experiment (the deli cookies and the Archway are soft).

COOKIE DATA SHEET -- Your filling out of this form indicates consent to participate in the experiment. NAME: ________________________________________________ 1. I buy a package of commercial chocolate chip cookies on the average about: (a) once per week, (b) less than once a week but at least once per month (c) less than once per month CIRCLE ONE 2. My preferred brand of chocolate chip cookies is: (Put "no preference" in appropriate). 3. I am MALE FEMALE (cirle one) RATINGS (circle) according to: P = poor F = fair G = good VG = Very good E = excellent ID TASTE TEXTURE APPEARANCE OVERALL COMMENTS A P F G VG E P F G VG E P F G VG E P F G VG E B P F G VG E P F G VG E P F G VG E P F G VG E C P F G VG E P F G VG E P F G VG E P F G VG E D P F G VG E P F G VG E P F G VG E P F G VG E E P F G VG E P F G VG E P F G VG E P F G VG E F P F G VG E P F G VG E P F G VG E P F G VG E G P F G VG E P F G VG E P F G VG E P F G VG E H P F G VG E P F G VG E P F G VG E P F G VG E \PRE