TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
An instructor says AI tools can complete take-home assignments in his upper-level Machine Learning in Production course, prompting changes to how students demonstrate learning. The course now relies more on live teaching-assistant conversations, exams and video demonstrations, while still allowing AI use in many settings.
An instructor of an upper-level Machine Learning in Production course says AI tools can now complete its take-home assignments, leading him to shift assessment toward teaching-assistant conversations, exams and video demonstrations. In a report for The Last Software Engineer?, he says the course’s goals have changed little: students should still learn to make engineering tradeoffs, anticipate risks and work in teams.
The course typically enrolls 100 to 170 students and focuses on building production-ready software around machine-learning models. The instructor says students may use AI in most course settings, without attribution, and that the course encourages AI use in some activities. Written and oral exams are exceptions. He says policing AI use is not feasible and students need to learn to use the technology responsibly.
One major change is that written responses no longer serve as evidence of students’ understanding. Reports may still describe a solution and point to relevant code, but the instructor says they are meant to help readers navigate the work and can be AI-generated. Written reflections and reading quizzes, including prompts about teamwork or data-quality problems, have been replaced or supplemented with 15-minute meetings with a teaching assistant after each assignment.
Those meetings count for 20% of assignment points and are graded pass or fail. Students who fail can try again. The instructor says TAs are encouraged to apply fairly high standards and that quite a few students fail on their first attempt. Other assessment methods include exams and video demonstrations. The source excerpt does not detail the format or weighting of those components.
Live Checks Reshape Course Assessment
The changes show how AI can alter the evidence educators use to judge learning, even when the intended learning outcomes stay stable. If students can delegate a take-home answer to a model, a polished submission may say less about what they understand or can explain. The instructor’s response is to ask students to discuss their work directly with a TA and defend or expand on their answers in conversation.
That response brings a tradeoff. Individual meetings take staff time, and the course shifts some weight from repeated, low-stakes work toward more supervised assessment. The instructor acknowledges that this runs against teaching practices that favor frequent assessments with feedback. He says he still sees value in repeated low-stakes interactions, even though some students may use AI to bypass the learning they are meant to support.
The account is one instructor’s description of a particular advanced course, not evidence that the same approach works across subjects. It matters to educators because it makes the practical problem specific: preserving learning goals while changing how students show they have met them.
From GPT-3 to AI Agents
The instructor traces the shift to a test around 2021, before ChatGPT launched. Vincent Hellendoorn suggested trying GPT-3 on the course’s reading quizzes. According to the report, the system produced answers that passed the grading rubric without seeing the assigned paper. The instructor says he made no changes at the time.
He says the pressure to redesign assessments grew as newer AI tools and coding agents became capable of completing more course work. The course has adjusted some topics as AI tools have changed, but its focus on engineering decisions, risk management and teamwork has largely remained in place. The instructor notes that introductory courses or traditional software-engineering courses might need to revisit their learning goals more substantially.
The account also describes changes to resubmissions. The course previously let students correct mistakes and regain a limited number of points, an approach intended to support learning from feedback. The instructor says some students began submitting AI-generated work first and consulting feedback only when preparing a resubmission. The course now applies a 10% penalty to resubmissions, according to the report.
“I have given up all parts of assignments that required a written text answer.”
— The course instructor, in The Last Software Engineer?
Questions About Learning and Workload
The report does not provide outcome data showing whether the changes have improved learning, reduced AI delegation or changed grades and completion rates. It also gives no comparison between student performance under the old and new assessments. The course instructor describes his experience but does not report results from a controlled evaluation.
Some practical details remain unspecified, including how the course uses exams and video demonstrations, how TA meetings are staffed across a class of up to 170 students, and how consistently different TAs apply the pass-or-fail standard. The report also does not say how students view the added meetings or whether the 10% resubmission penalty has changed the pattern of AI use.
Further Evaluation Could Clarify Results
The instructor says Stanford’s CS221 course is evaluating a similar approach to live student conversations in a controlled experiment during the semester covered by his account. The report provides no results or publication date for that experiment, so its findings remain pending in the source material.
For the Machine Learning in Production course, the instructor describes the assessment changes as current practice but does not announce a next milestone or a planned review. More information about student learning, TA workload and the outcomes of the Stanford evaluation would help establish how well live conversations work alongside take-home assignments in courses that permit AI use.
Key Questions
What changed in the course?
The instructor says he removed written text answers as a way to assess understanding. Students now have 15-minute conversations with a TA after each assignment, while exams and video demonstrations are also part of the course’s assessment approach.
Are students allowed to use AI?
According to the instructor, students may use AI in most course settings, without attribution. Written and oral exams are exceptions, and the course encourages AI use in some activities.
Why add meetings with teaching assistants?
The instructor says written reflections and reading-quiz responses could be delegated to AI. In a live conversation, a TA can ask follow-up questions about a student’s work and understanding.
Do the changes preserve the course’s learning goals?
The instructor says the course’s goals have barely changed. They remain focused on engineering tradeoffs, anticipating and mitigating risks, and teamwork rather than specific tools or coding tasks.
Is there evidence the new approach works?
The report describes the instructor’s experience but gives no outcome data comparing learning before and after the changes. It says Stanford’s CS221 course is evaluating a similar live-conversation approach, but reports no results.
Source: hn
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
