Submission Deadline Marks and Feedback Before 10am on: Tue 17/12/2019 Fri 20/12/2019 (if required an extension could be granted by Student Mitigation Team) 20 working days after deadline (L4, 5 and 7)

Submission Deadline

Marks and Feedback

Before 10am on:

Tue 17/12/2019 Fri 20/12/2019 (if required an extension could be granted by Student Mitigation Team)

20 working days after deadline (L4, 5 and 7)

15 working days after deadline (L6)

10 working days after deadline (block delivery)

Unit title & code

CIS111-6: Intelligent Systems and Data Mining

Assignment number and title

Assignment 1: Data Mining Solutions for Direct Marketing Campaign

Assessment type

WR

Weighting of assessment

50%

Unit learning outcomes

Analyse a Data Mining technique capable of supporting practitioners to make reliable decisions which require predictive modelling, for example, in a Business scenario

Demonstrate results of using an efficient technique which is capable of finding a solution to a given predictive problem represented by a data set

Evaluate the accuracy of the technique in terms of differences between the predicted values and the given data

What am I required to do in this assignment?

1. Task

Students will develop a DM solution for saving cost of a direct marketing campaign by reducing false positive (wasted call) and false negative (missed customer) decisions. Working on this assignment, students can consider the following scenario.

A Bank has decided to save the cost of a direct marketing campaign based on phone calls offering a product to a client. A cost efficient solution is expected to support the campaign with predictions for a given client profile whether the client subscribes to the product or not. A startup company wants to develop an innovative DM technology which will be competitive on the market. The Manager will interview and hire Data Analysts. The team will analyse the existing technologies to design a DM solution winning the competition. A team Manager will choose the best solution for the market competition in terms of cost efficiency. The evaluation of the developed solutions will be made on the test data. The costs will be defined for both the false positive and false negative predictions.

Examples of cost-efficient DM solutions for direct marketing are provided on the UCI Machine Learning repository describing a Bank Marketing problem.

Students will apply for one of roles: (i) group manager, (ii) group member, or will work  individually. The group manager will arrange comparison and ranking of solutions designed in a group, and will have additional 5 points. Each student will run individual experiments to find an efficient solution and  describe differences in experimental results.

2. Method and Technology

To design a solution, students will use Data Mining techniques such as Decision Trees and Artificial Neural Networks. Examples of solutions will be provided in R Scripting using (i) a Cloud technology CoCalc or (ii) an advanced development suit RStudio free for students.

3. Data        

The Assignment 1 Bank Marketing data set is available as a csv file.

4. Report submission and report template

Each solution will be evaluated in terms of the costs of false decisions made on the validation data. Reports will be submitted via BREO.  Reports can be prepared with a template. BREO similarity level of reports must be < 20%.

Is there a size limit?

2000 words on average

What do I need to do to pass? (Threshold Expectations from UIF)

Apply Decision Tree technique to solve the Bank Marketing task presented by a set of customer profiles

Analyse problems which are required to be resolved in order to develop a solution providing a high prediction accuracy on a given data set.

How do I produce high quality work that merits a good grade?

Identify a set of parameters which are required to be adjusted within DM techniques in order to optimise the solution in terms of prediction accuracy

Explain how the parameters of a DM technique influence the prediction accuracy

Run experiments in order to verify the solution designed on the given data set

Analyse and compare the results of the experiments in a group and with the known from the literature.

How does assignment relate to what we are doing in scheduled sessions?

Data Mining techniques and use cases developed in R will be considered during  lectures and tutorials.

How will my assignment be marked?

Your assignment be marked according to the threshold expectations and the criteria on the following page.

You can use them to evaluate your own work and estimate your grade before you submit.

#

Weight, %

Lower 2nd – 50-59%

Upper 2nd – 60-69%

1st Class – 70%+

1

Analysis

(20)

Fair analysis of the basic approaches

Relatively good analysis of the relevant literature, mainly covering the state-of-art

Excellent analysis of the relevant literature, fully covering the state-of-art

2

Design

(50)

Fair design of a basic solution providing a reasonable performance within a single set of parameters

Design of a solution providing a fair performance in a series of experiments with different sets of parameters

Design of a solution providing a performance, competitive to known from the literature, in a series of experiments with different sets of parameters

3

Conclusion (30)

Fair conclusion on the experimental results obtained within a single set of parameters

Conclusion on and comparison of the experimental results obtained within two different sets of parameters

Conclusion on and comparison of the experimental results obtained within multiple sets of parameters, demonstrating a solution which provides a competitive performance