====== Autograding Jupyter Notebook assignments with Otter - quickstart ======

Otter uses raw notebook cells, together with the first comment line in each of these, for delimitation between blocks, i.e. where questions, tests or configurations begin and/or end. Ex:
<code># BEGIN SOLUTION</code>
<code>
def my_func(a, b):
	# BEGIN SOLUTION
	c = (a+b)/2 
	# END SOLUTION
return c
</code>
<code># END SOLUTION</code>
In the above, a solution block is started and ended with the first and last lines, as they each lead a raw cell. Inside my_func, # BEGIN SOLUTION and # END SOLUTION are used to hide a specific part of code from the notebooks generated for students(but they still belong to the same block as they’re not in sperate raw cells).

=====Creating assignments=====
The first cell shall be a raw cell containing a YAML-formatted configuration block. Use ''# ASSIGNMENT CONFIG'' to de fine this. This specifies how the assignments should be generated , tests run, etc. [[https://otter-grader.readthedocs.io/en/latest/otter_assign/notebook_format.html|Find more details directly from Otter here]] or [[otter-grader|on this wiki here]]. 
Example:
<code>
# ASSIGNMENT CONFIG
requirements: requirements.txt
solutions_pdf: true
export_cell:
    instructions: "These are some submission instructions."
generate: 
    pdf: true
    filtering: true
    pagebreaks: true
    zips: false
</code>
To write question descriptions you can use a simple markdown cell, no additional syntax required \\
====Question blocks====
Use ''# BEGIN QUESTION'' and ''# END QUESTION'' to define a question block. Within it, use ''# BEGIN SOLUTION'', ''# END SOLUTION'' to specify the answer, and ''# BEGIN TESTS'' and ''# END TESTS'' to specify tests. Example:
<code>
# BEGIN QUESTION
name: q1
points: 2
</code>
<code>
Question 1. Write a function called sieve that takes in a positive integer n and returns a set of the prime numbers less than or equal to n. Use the Sieve of Eratosthenes to find the primes.
# BEGIN SOLUTION
def sieve(n):
    """
    Generate a set of prime numbers less than or equal to a positive integer.
    """
    # BEGIN SOLUTION
    is_prime = [True for _ in range(n + 1)]
    p = 2
    while p ** 2 <= n:
        if is_prime[p]:
            for i in range(p ** 2, n + 1, p):
                is_prime[i] = False
        p += 1

    is_prime[0]= False
    is_prime[1]= False

    return set(i for i in range(n + 1) if is_prime[i])
    # END SOLUTION
# END SOLUTION
</code>
<code>
# BEGIN TESTS
</code>
<code>
def test_low_primes(sieve):
    assert sieve(1) == set()
    assert sieve(2) == {2}
    assert sieve(3) == {2, 3}

test_low_primes(sieve)  # IGNORE
# HIDDEN
def test_higher_primes(sieve):
    assert sieve(20) == {2, 3, 5, 7, 11, 13, 17, 19}
    assert sieve(100) == {2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97}

test_higher_primes(sieve)  # IGNORE
</code>
<code>
# END TESTS
</code>
<code>
# END QUESTION
</code>


As seen above, you can specify a name and points for a question after declaring its block. Same can be done for tests. [[https://otter-grader.readthedocs.io/en/latest/otter_assign/notebook_format.html#autograded-questions|See here for details]]
**Generate assignment**
Once finished writing the assignment, you can generate the student version, along with the autograding zip, PDF and other files with:
<code>
otter assign <loc-to-notebook> <loc-to-save>
</code>
Where the first argument is the filepath to your Notebook and the second is the location where you want the generated files to be saved.
=====Autograding assignments=====
Open a terminal(in Jupyter: File->New Launcher->Other->Terminal), go to where you have saved assignment_grader.py and run:
<code>
python assignment_grader.py  -a <path-to-autograder-zip> -s <path-to-student-submissions> -c <where-to-save-csv> -tp <whether-to-generate-feedback-pdfs> [0,1] -p <where-to-save-pdfs>
</code>
Each argument is optional, the script looks into or creates default locations for all its arguments. Feedback is generated by default(tp=1). \\
Example:
<code>
python assignment_grader.py  -a ml_hw1/dist/autograder/autograder.zip -s ml_hw1/submissions -c ml_hw1/results -tp 1 -p /ml_hw1/test_feedback
</code>