{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c00bf530",
   "metadata": {},
   "source": [
    "# Project 2: Classification\n",
    "\n",
    "This project asks you to perform various experiments with classification. The dataset we are using is a toy dataset for credit card fraud detection:\n",
    "\n",
    "https://www.kaggle.com/datasets/shubhamjoshi2130of/abstract-data-set-for-credit-card-fraud-detection\n",
    "\n",
    "You will write code and discussion texts into code and text cells in this notebook. \n",
    "\n",
    "If a block starts with TODO:, this means that you need to write something there. \n",
    "\n",
    "Some code had been written for you to guide the project. Don't change the already written code.\n",
    "\n",
    "## Grading\n",
    "The points add up to 40, that is 30 + 10 bonus points. While there is no difference between the regular and the bonus points, I recommend that you solve the problems labeled \"BONUS\" after you finished the other ones. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a440a191",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import scipy as sp\n",
    "import sklearn as sk"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "691d1a9f",
   "metadata": {},
   "source": [
    "## Setup for the project\n",
    "\n",
    "Here we load the dataset, and create the training and test datasets as numpy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "902ccefe",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"creditcard.csv\",  true_values=[\"Y\"], false_values=[\"N\"])\n",
    "pd.show_versions()\n",
    "print(f\"Number of rows {len(df.index)}\")\n",
    "print(f\"The columns of the database {df.columns}\")\n",
    "df.value_counts(\"isFradulent\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54bf2ec2",
   "metadata": {},
   "outputs": [],
   "source": [
    "xfields = [\n",
    "    'Average Amount/transaction/day',\n",
    "       'Transaction_amount', 'Is declined', 'Total Number of declines/day',\n",
    "       'isForeignTransaction', 'isHighRiskCountry', 'Daily_chargeback_avg_amt',\n",
    "       '6_month_avg_chbk_amt', '6-month_chbk_freq']\n",
    "\n",
    "df_shuffled = df.sample(frac=1) # shuffle the rows\n",
    "x = df_shuffled[xfields].to_numpy(dtype=np.float64)\n",
    "y = df_shuffled[\"isFradulent\"].to_numpy(dtype=np.float64)\n",
    "# the training data is the first 2000 rows, after shuffled\n",
    "training_data_x = x[:2000]\n",
    "training_data_y = y[:2000]\n",
    "# the test data is the remaining\n",
    "test_data_x = x[2000:]\n",
    "test_data_y = y[2000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "959fcb7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Run this to help you with what number goes with what field:\")\n",
    "for i, x in enumerate(xfields):\n",
    "    print(f\"{i} = {x}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba69479d",
   "metadata": {},
   "source": [
    "## P1: Create an accuracy metric (7 pts)\n",
    "Create a simple accuracy metric function which for a pair of ground truth values $y$ and estimates $\\hat{y}$ (both of them arrays) calculates the accuracy of the estimate $\\hat{y}$. For instance, if you pass y = [1, 0, 1] and \n",
    "yhat = [1, 1, 0], the loss function should return 0.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9bd1d0ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "def accuracy(y, yhat):\n",
    "    ## implement here\n",
    "    return 0.0\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a577d090",
   "metadata": {},
   "outputs": [],
   "source": [
    "# test your function here\n",
    "acc = accuracy([1, 0, 1], [1, 1, 0])\n",
    "print(f\"Accuracy is {acc}\") # should print 0.33..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "564999e3",
   "metadata": {},
   "source": [
    "## P2: Implement a majority classifier (7 pts)\n",
    "This classifier will always return the most likely value. Training the classifier means determining what is the most likely value (regardless vhat value you pass to it). For instance, if more than half of the transactions are fraudulent, then you just return fraudulent always. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7207afba",
   "metadata": {},
   "outputs": [],
   "source": [
    "def classify_majority(x, theta):\n",
    "    # whatever the value of x, we return the theta\n",
    "    return theta\n",
    "\n",
    "# TODO: implement the train majority function\n",
    "def train_majority(training_x, training_y):\n",
    "    # this function will have to determine which is more likely to \n",
    "    # be the value of y, one (true) or zero (false)\n",
    "    return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d61379a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: use the train_majority function to find the theta value for the training dataset\n",
    "theta = 0\n",
    "\n",
    "# TODO: now use the theta value to create the test_data_yhat array which contains the classification for each test value \n",
    "\n",
    "# TODO: now calculate the accuracy of the classifier using the function implemented in P1, and print it out\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc53d27d",
   "metadata": {},
   "source": [
    "TODO: Discuss here the performance of the majority classifier. Would this beat a classifier that just returns random values? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c63bf75",
   "metadata": {},
   "source": [
    "## P3: Implement a hand engineered classifier (8 pts)\n",
    "\n",
    "Engineer by hand a classifier function that predicts whether  a transaction is  fraudulent or not. Your function should have a $\\theta$ parameter which allows you to tweak it. \n",
    "The problem requires you to design a function that performs this classification, tweak its parameters, and measure its accuracy for the best parametrization you found. You should aim for a function that, at minimum, performs better than the majority classifier. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66d0d477",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: implement here your hand-engineered classifier\n",
    "# The example below is just a very bad example, but it gives you an idea of how you can reason about the classification problem.\n",
    "# In your implementation, you should try to actually find some kind of clever algorithm. You can also use more complex parametrizations\n",
    "\n",
    "def classify_handwritten(x, theta):\n",
    "    \"\"\"Fraudulent transaction classifier. In this test example, we classify every foreign transaction as fraudulent. Transactions larger than theta[0] are also fraudulent\"\"\"\n",
    "    if x[5] == 1:\n",
    "        return 1\n",
    "    else:\n",
    "        if x[4] == 1:\n",
    "            return 1\n",
    "    if x[1] > theta[0]:\n",
    "        return 1\n",
    "    return 0\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a723623d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Now, run some experiments with your function. Experiment with different values of the parameter theta.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86997918",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: calculate the accuracy of the classifier on the test data with the best\n",
    "# theta found above and print it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c319f058",
   "metadata": {},
   "source": [
    "TODO: Describe in one paragraph your experiments and evaluation. Discuss the overall accuracy your classifier. Did you manage to beat the \"majority\" classifier? Comment on how easy or hard was to do this. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0548dea",
   "metadata": {},
   "source": [
    "## P4: Implement a logistic regression classifier using sklearn (8 pts)\n",
    "Implement a logistic regression function using the sklearn library. \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b91956b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: implement the logistic regression here in a function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7af53924",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: now, run some experiments with it, and measure the accuracy with various parametrizations. In particular, you should run it with and without regularization. \n",
    "# In the last line, print the accuracy with the best parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20b3bfac",
   "metadata": {},
   "source": [
    "\n",
    "TODO: Describe in one paragraph your experiments and evaluation of the Logistic Regression classifier. Consider things such as accuracy, training time, ease of tweaking of the parameters. Compare it with the accuracy of the hand-engineered classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5fcd1c6",
   "metadata": {},
   "source": [
    "## P5 Bonus: Implement a random forest classifier using sklearn (5 pts)\n",
    "Implement a random forest classifier using sklearn \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9663e31d",
   "metadata": {},
   "outputs": [],
   "source": [
    " # TODO: Implement the random forest classifier here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65e10363",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Perform some experiments here with different parameters of the random forest classifier. In the last line, print the accuracy with the best parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a738769a",
   "metadata": {},
   "source": [
    "TODO: Describe in one paragraph your experiments and evaluation of the random forest classifier. Consider things such as accuracy, training time, ease of tweaking of the parameters. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c46a9f4",
   "metadata": {},
   "source": [
    "## P6 Bonus: Implement an AdaBoost classifer using sklearn (5 pts)\n",
    "\n",
    "Implement an AdaBoost classifier using sklearn https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc373403",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Implement the adaboost classifier here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "294dde6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Perform some experiments here with different parametrizations of the adaboost classifier. In the last line, print the accuracy with the best parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d1dd74e",
   "metadata": {},
   "source": [
    "TODO: Describe in one paragraph your experiments and evaluation of the AdaBoost classifier. Consider things such as accuracy, training time, ease of tweaking of the parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6cc7e2ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "# SOLUTION\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "clf = AdaBoostClassifier()\n",
    "clf.fit(training_data_x, training_data_y)\n",
    "yhat = clf.predict(test_data_x)\n",
    "acc = accuracy(test_data_y, yhat)\n",
    "print(f\"Accuracy of the AdaBoost classifier {acc}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.8 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "vscode": {
   "interpreter": {
    "hash": "13140b3b9092b9a26a4b55ddc500d8b0c9f21b15e8ef2dd16bed19d6074a1e03"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}