I’m at the STEAM ed Finland conference in Jyväskylä where Mariia Gavriushenko is talking about personalized learning environments.

Web-based learning systems are being more and more widely used in large part because they can be used any time, anywhere. She points to two types: Learning management systems and game-based systems. But they lack personalization that makes them suitable for particular learners in terms of learning speed, knowledge background, preferences in learning and career, goals for future life, and their differing habits. Personalized systems can provide assistance in learning and adapt their learning path. Web-based learning shouldn’t just be more convenient. It should also be better adapted to personal needs.

But this is hard. But if you can do it, it can monitor the learner’s knowledge level and automatically present the right materials. In can help teachers create suitable material and find the most relevant content and convert it into comprehensive info. It can also help students identify the best courses and programs.

She talks about two types of personalized learning systems: 1. systems that allow the user to change the system or 2. the sysytem changes itself to meet the users needs. The systems can be based on rules and context or can be algorithm driven.

Five main features of adaptive learning systems:

• Pre-test

• Pacing and control

• Feedback and assessment

• Progress tracking and reports

• Motivation and reward

The ontological presentation of every learner keeps something like a profile for each user, enabling semantic reasoning.

She gives an example of this model: automated academic advising. It’s based on learning analytics. It’s an intelligent learning support system based on semantically-enhanced decision support, that identifies gaps, and recommends materials and courses. It can create a personal study plan. The ontology helps the system understand which topics are connected to others so that it can identify knowledge gaps.

An adaptive vocabulary learning environment provides cildren with an adaptive way to train their vocabulary, taking into account the individuality of the learner. It assumes the more similar the words, the harder they are to recognize.

Mariia believes we will make increasing use of adaptive educational tech.

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Maarit Rossi, who teaches math teaching around the world, is talking on the topic: “AI forces us to change maths education.”

Finnish teachers are doing a great, great job, she says. “But we are doing it too quietly.”

Education is too similar to industrial assembly lines. Students sit passively in rows. Students find math education to be boring, meaningless, and frightening. Typically this happens sometime in 5-7th grade. Teaching math has not changed in 100 years. It is a global problem.

Meanwhile, tech is changing really quickly. (She shows a photo from 1956 of workers shoving a 5 megabyte drive onto a truck.)

These days we are talking about personalizing math education. Easily available programs solve math problems. In the USA, people say the students are “cheating.” No, they’re being educated wrong. We need to be asking if we’re teaching students 10 critical skills, including cognitive flexibility, nebotiation, coordinating with others, emotional intelligence, critical thinking, creaetivity, complex problem solving, service orientation [and a couple of others I didn’t have time to copy down].

A modern math curriculum addresses attitudes, metacognition (e.g., self-regulation), skills, concepts, and processes. Instead, we focus on the concepts (e.g., algebraic, statistical, etc.).

A classroom has to be a safe place where you can make mistakes.

There are four pillars: practice, learning by doing, social learning, and interdisciplinary math. She gives some examples. Students estimate the price of a week’s shopping for a family of four. Maaritt has students work in groups of four. After that, they go to the nearest shop to find the actual prices; the students have to divide up the task to get it done in time. (You can have them do online shopping if there isn’t nearby shop.) Students estimate and round the numbers, tasks that are usually taught separately.

For higher grades, the students deal with real data from an African refugee camp. The students have to estimate how much food is needed to keep everyone alive for two weeks. “This is meaningful to them.”

It’s important for math to have double the lesson length. If it’s only one hour, it is not enough. “The students love it when they have the opportunity to think, to discover, to find themselves.”

Re-arrange the classroom. Cluster the tables rather than rows. The students can teach one another. “It is important that the feel successful.”

“And of course we use computers. And apps. And phones.”

“Math is also interesting because it can model many things.” If they have an embodied sense of a cubic meter, for example, they learn how to convert them to other measures. Or model the size of the solar system outside.

She has students estimate collections of objects, e.g. a bowl of noodles. Then they round. Then they count. Groups come up with strategies for counting, including doing it in ways that enable the count to be interrupted and resumed.

Physical exercise makes brains work better.

Classifying is important. She asks students to take sheets of paper and make the biggest triangle they can, and another of a different shape. They put all the triangles in the middle of the room. Then she asks them to see if they can cluster them by similarities.

“Students need to use their own language” rather than only hearing the teacher talk. This is how they learn to understand.

[My notes about the last few minutes, and the questions, go cut off via brain-computer glitch. Sorry.]

TAGS:

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Maria Kankaanranta, Leena Hiltunen, Kati Clements and Tiina Mäkelä are on the faculty of the School of Education at the University of Jyväskylä The are going to talk about SMART education.

SMART means self-directed, motivated, adaptive, reseource enriched, and technology-embedded learning. (They credit South Korean researchers for this.) This is a paradigm shift: From education a specific times to any time. From lectures to motivated ed methods. From teaching the 3Rs to epanding the ed capacity. From traditional textbooks to enriched resources. From a physical space to anywhere there is the enabling tech.

One project (Horizon 2020) works across disciplines to connect students, parents, teachers, and companies. Companies expect universities to develop the skills they need, but you really have to begin with primary school. The aim of the project is to create a pedagogical framework and design principles for attractive and engaging STEM learning environments. She presents a long list of pedagogical design principles that guide the design of this kind of hybrid learning enviroments. It includes adaptive learning, self-regulation, project-based learning, novelty, but also conventionality: “you don’t have to abandon everything.”

What beyond MOODLE can we do? The EU has funded instruments for procurement of innovation. The presenters have worked on IMAILE & LEA (LearnTech Accelerator). IMAILE ran for 48 months in four countries. To address problems, the project pointed to two existing solutions: YipTree and AMIGO (e-books publisher from Spain). YipTree provides individual personalized learning paths (adaptive materials), student motivation by a virtual tutor and by other students, gamificiation, quick assessment tools, and notifications when a student is having difficulties. They tested this in two schools per country. YipTree did well.

They have been training teachers in computational thinking, programming, and robotics. They use online, mobile apps to make it available and free for all teachers and students. They’re using different training models to motivate and encourage teachers to adopt these apps. E.g., they’re “hijacking” schools and workplaces to train them where they are. Teachers really want human engagement.

Schools have access to tech resources but they’re under-used because the teachers don’t know what’s available and possible. This presentation’s project is helping teachers with this.

Conclusion: Smart ed is not easy. It takes time. It requires getting out of your comfort zone. It requires training, tools, research, and a human touch.

Q&A

Q: Does your model take into account students with disabilities?

A: Yes. Part of this is “access for all.” Also, IMAILE does. Imperfectly. They collaborate with a local school for the impaired.

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä where Ulla Richardson is going to talk about GraphoLearn, an adaptive learning method for learning to read.

Ulla has been working on the Jyväskylä< Longitudinal Study of Dyslexia (JLD). Globally, one third of people can’t read or have poor reading skills. One fifth of Europe also. About 15% of children have learning disabilities.

One Issue: knowing which sound goes with which letters. GraphoLearn is a game to help students with this, developed by a multidisciplinary team. You learn a word by connecting a sound to a written letter. Then you can move to syllables and words. The game teaches by trial and error. If you get it wrong, it immediately tells you the correct sound. It uses a simple adaptive approach to select the wrong choices that are presented. The game aims at being entertaining, and motivates also with points and rewards. It’s a multi-modal system: visual and audio. It helps dyslexics by training them on the distinctions between sounds. Unlike human beings, it never displays any impatience.

It adapts to the user’s skill level, automatically assessing performance and aiming at at 80% accuracy so that it’s challenging but not too challenging.

13,000 players have played in Finland, and more in other languages. Ulla displays data that shows positive results among students who use GraphoLearn, including when teaching English where every letter has multiple pronunciations.

There are some difficulties analyzing the logs: there’s great variability in how kids play the game, how long they play, etc. There’s no background info on the students. [I missed some of this.] There’s an opportunity to come up with new ways to understand and analyze this data.

Q&A

Q: Your work is amazing. When I was learning English I could already read Finnish, so I made natural mispronunciations of ape, anarchist, etc. How do you cope with this?

A: Spoken and written English are like separate languages, especially if Finnish is your first language where each letter has only one pronunciation. You need a bigger unit to teach a language like English. That’s why we have the Rime approach where we show the letters in more context. [I may have gotten this wrong.]

Q: How hard is it to adapt the game to each language’s logic?

Understanding xanax: A Prescription Medication

xanax, a brand name for alprazolam, is a prescription benzodiazepine used to treat anxiety and panic disorders. Here’s what you need to know:

• Typically taken orally in tablet form

• Dosage varies based on individual needs and doctor’s prescription

• Can be taken with or without food

• Usually administered 2-4 times daily

Remember: xanax should only be taken as prescribed by a healthcare professional. Misuse can lead to dependence or serious side effects.

Have questions about xanax management? Consult your doctor for safe, personalized treatment

A: It’s hard.

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Mirka Saarela and Sanna Juutinen are talking about their analysis of education data.

There’s a triennial worldwide study by the OECD to assess students. Usually, people are only interested in its ranking of education by country. Finland does extremely well at this. This is surprising because Finland does not do particularly well in the factors that are taken to produce high quality educational systems. So Finnish ed has been studied extensively. PISA augments this analysis using learning analytics. (The US does at best average in the OECD ranking.)

Traditional research usually starts with the literature, develops a hypothesis, collects the data, and checks the result. PISA’s data mining approach starts with the data. “We want to find a needle in the haystack, but we don’t know what the needle looks like.” That is, they don’t know what type of pattern to look for.

Results of 2012 PISA: If you cluster all 24M students with their characteristics and attitudes without regard to their country you get clusters for Asia, developing world, Islamic, western countries. So, that maps well.

For Finland, the most salient factor seems to be its comprehensive school system that promotes equality and equity.

In 2015 for the first time there was a computerized test environment available. Most students used it. The logfile recorded how long students spent on a task and the number of activities (mouse clicks, etc.) as well as the score. They examined the Finnish log file to find student profiles, related to student’s strategies and knowledge. Their analysis found five different clusters. [I can’t read the slide from here. Sorry.] They are still studying what this tells us. (They purposefully have not yet factored in gender.)

Nov. 2017 results showed that girls did far better than boys. The test was done in a chat environment which might have been more familiar for the girls? Is the computerization of the tests affecting the results? Is the computerization of education affecting the results? More research is needed.

Q&A

Q: Does the clustering suggest interventions? E.g., “Slow down. Less clicking.”

A: [I couldn’t quite hear the answer, but I think the answer is that it needs more analysis. I think.]

Q: I work for ETS. Are the slides available?

A: Yes, but the research isn’t public yet.

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Harri Ketamo is giving a talk on “micro-learning.” He recently won a prestigious prize for the best new ideas in Finland. He is interested in the use of AI for learning.

We don’t have enough good teachers globally, so we have to think about ed in new ways, Harri says. Can we use AI to bring good ed to everyone without hiring 200M new teachers globally? If we paid teachers equivalent to doctors and lawyers, we could hire those 200M. But we apparently not willing to do that.

One challenge: Career coaching. What do you want to study? Why? What are the skills you need? What do you need to know?

His company does natural language analysis — not word matches, but meaning. As an example he shows a shareholder agreement. Such agreements always have the same elements. After being trained on law, his company’s AI can create a map of the topic and analyze a block of text to see if it covers the legal requirements…the sort of work that a legal assistant does. For some standard agreements, we may soon not need lawyers, he predicts.

The system’s language model is a mess of words and relations. But if you zoom out from the map, the AI has clustered the concepts. At the Slush Sanghai conference, his AI could develop a list of the companies a customer might want to meet based on a text analysis of the companies’ web sites, etc. Likewise if your business is looking for help with a project.

Finland has a lot of public data about skills and openings. Universities’ curricula are publicly available.[Yay!] Unlike LinkedIn, all this data is public. Harri shows a map that displays the skills and competencies Finnish businesses want and the matching training offered by Finnish universities. The system can explore public information about a user and map that to available jobs and the training that is required and available for it. The available jobs are listed with relevancy expressed as a percentage. It can also look internationally to find matches.

The AI can also put together a course for a topic that a user needs. It can tell what the core concepts are by mining publications, courses, news, etc. The result is an interaction with a bot that talks with you in a Whatsapp like way. (See his paper “Agents and Analytics: A framework for educational data mining with games based learning”). It generates tests that show what a student needs to study if she gets a question wrong.

His newest project, in process: Libraries are the biggest collections of creative, educational material, so the AI ought to point people there. His software can find the common sources among courses and areas of study. It can discover the skills and competencies that materials can teach. This lets it cluster materials around degree programs. It can also generate micro-educational programs, curating a collection of readings.

His platform has an open an API. See Headai.

Q&A

Q: Have you done controlled experiments?

A: Yes. We’ve found that people get 20-40% better performance when our software is used in blended model, i.e., with a human teacher. It helps motivate people if they can see the areas they need to work on disappear over time.

Q: The sw only found male authors in the example you put up of automatically collated materials.

A: Small training set. Gender is not part of the metadata in Finland.

A: Don’t you worry that your system will exacerbate bias?

Q: Humans are biased. AI is a black box. We need to think about how to manage this

Q: [me] Are the topics generated from the content? Or do you start off with an ontology?

A: It creates its ontology out of the data.

Q: [me] Are you committing to make sure that the results of your AI do not reflect the built in biases?

A: Our news system on the Web presents a range of views. We need to think about how to do this for gender issues with the course software.

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Geun-Sik Jo is a professor at Inha University in Seoul. He teaches AI and is an augmented reality expert. He also has a startup using AR for aircraft maintenance [pdf].

Why do we need mashups? To foster innovation. To create new systems very efficiently. He uses the integration of Google Maps into Craigslist as a simple example.

Interactive video — video with clickable spots — is often a mashup: click and go to a product page or send a text to a friend. E.g., https://www.wirewax.com or http://www.raptmedia or http://www.zentrick.com
http://www.zentrick.com .

TVs today are computers with their own operating systems and applications. Prof. Jo shows a video of AR TV. The “screen” is a virtual image displayed on special glasses.

If we mash up services, linked data clouds, TV content, social media, int an AR device, “we can do nice things.”

He shows some videos created by his students that present AR objects that are linked to the Internet: a clickable travel ad, location-based pizza ordering, a very cool dance instruction video, a short movie.

He shows a demo of the AI Content Creation System that can make movies. In the example, it creates a drama, mashing up scenes from multiple movies. The system identifies the characters, their actions, and their displayed emotions.

Is this creative? “I’m not a philosopher. I’m explaining this from the engineering point of view,” he says modestly. [If remixes count as creativity — and I certainly think they do — then it’s creative. Does that mean the AI system is creative? Not necessarily in any meaningful sense. Debate amongst yourselves.]

Be the first to comment »

I’m at the STEAM ed Finland conference in Jyväskylä. Sonja Amadae at Swansea University (also currently at Helsinki U.) works on robotic ethics. She will argue in this talk that computers are algorithmic, that they only do what they’re programmed to do, that they don’t understand what they’re doing and they don’t feel human experience. AI is, she concludes, a tool.

AI is like a human prosthetic that helps us walk. AI is an enhancement of human capabilities.

She will talk about about three cases.

Case 1: Generating a Rembrandt

A bank funded a project [cool site about it] to see what would happen if a computer had all of the data about Rembrandt’s portraits. They quantified the paintings: types, facial aspects including the size and distance of facial features, depth, contour, etc. They programmed the algorithm to create a portrait. The result was quite goood. People were pleased. Of course, it painted a white male. Is that creativity?

We are now recogizing the biases widespread in AI. E.g., “Biased algorithms are everywhere and one seeems to care” in MIT Tech Review by Will Knight. She also points to the time that Google mistakenly tagged black people as “gorillas.” So, we know there are limitations.

So, we fix the problem…and we end up with facial recognition systems so good that China can identify jaywalkers from surveillance cams, and then they post their images and names on large screens at the intersections.

Case 2: Forgery detection

The aim of one project was to detect forgeries. It was built on work done by Marits Michel van Dantzig in the 1950s. He looked at the brushstrokes on a painting; artists have signature brushstrokes. Each painting has on average 80,000 brushstrokes. A computer can compare a suspect painting’s brushstrokes with the legitimate brushstrokes of the artist. The result: the AI could identify forgeries 80% of the time from a single stroke.

Case 3: Computational creativity

She cites Wikipedia on Computational Creativity because she thinks it gets it roughly right:

Computational creativity (also known as artificial creativity, mechanical creativity, creative computing or creative computation) is a multidisciplinary endeavour that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, and the arts.
The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:[

• To construct a program or computer capable of human-level creativity.

• To better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans.

• To design programs that can enhance human creativity without necessarily being creative themselves.

She also quotes John McCarthy:

‘To ascribe certain beliefs, knowledge, free will, intentions, consciousness, abilities, or wants to a machine or computer program is legitimate when such an ascription expresses the same information about the machine that it expresses about a person.’

If you google “computational art” and you’ll see many pictures created computationally. [Or here.] Is it genuine creativity? What’s going on here?

We know that AI’s products can be taken as human. A poem created by AI won a poetry contest for humans. E.g., “A home transformed by the lightning the balanced alcoves smother” But the AI doesn’t know it’s making a poem.

Can AI make art? Well, art is in the eye of the beholder, so if you think it’s art, it’s art. But philosophically, we need to recall Turing-Church Computability which states that the computation “need not be intelligible to the one calculating.” The fact that computers can create works that look creative does not mean that the machines have the awareness required for creativity.

Can the operations of the brain be simulated on a computer? The Turing-Church statement says yes. But now we have computing so advanced that it’s unpredictable, probabilistic, and is beyond human capability. But the computations need not be artistic to the one computing.

Computability has limits:

1. Information and data are not meaning or knowledge.

2. Every single moment of existence is unique in the universe. Every single moment we see a unique aspect of the world. A Turing computer can’t see the outside world. It only has what it’s internal to it.

3. Human mind has existential experience.

4. The mind can reflect on itself.

5. Scott Aaronson says that humans can exercise free will and AI cannot, based on quantum theory. [Something about quantum free states.]

6.The universe has non-computable systems. Equilibrium paths?

“Aspect seeing” means that we can make a choice about how we see what we see. And each moment of each aspect is unique in time.

In SF, the SPCA uses a robot to chase away homeless people. Robots cannot exercise compassion.

Computers compute. Humans create. Creativity is not computable.

Q&A

Q: [me] Very interesting talk. What’s at stake in the question?

A: AI has had such a huge presence in our lives. There’s a power of thinking about rationality as computation. Gets best articulated in game theory. Can we conclude that this game theoretical rationality — the foundational understanding of rationality — is computable? Do human brings anything to the table? This leads to an argument for the obsolescence of the human. If we’re just computational, then we aren’t capable of any creativity. Or free will. That’s what’s ultimately at stake here.

Q: Can we create machines that are irrational, and have them bring a more human creativity?

A: There are many more types of rationality than game theory sort. E.g., we are rational in connection with one another working toward shared goals. The dichotomy between the rational and irrational is not sufficient.
TAGS:

Be the first to comment »

I’ve been at a two-day workshop sponsored by the Michigan State Uiversity and the National Science Foundation: “Workshop on Trustworthy Algorithmic Decision-Making.” After multiple rounds of rotating through workgroups iterating on five different questions, each group presented its findings — questions, insights, areas of future research.

Seriously, I cannot capture all of this.

Conduct of Data Science

What are the problems?

• Who defines and how do we ensure good practice in data science and machine learning?

Why is the topic important? Because algorithms are important. And they have important real-world effects on people’s lives.

Why is the problem difficult?

• Wrong incentives.

• It can be difficult to generalize practices.

• Best practices may be good for one goal but not another, e.g., efficiency but not social good. Also: Lack of shared concepts and vocabulary.

How to mitigate the problems?

• Change incentives

• Increase communication via vocabularies, translations

• Education through MOOCS, meetups, professional organizations

• Enable and encourage resource sharing: an open source lesson about bias, code sharing, data set sharing

Accountability group

The problem: How to integratively assess the impact of an algorithmic system on the public good? “Integrative” = the impact may be positive and negative and affect systems in complex ways. The impacts may be distributed differently across a population, so you have to think about disparities. These impacts may well change over time

We aim to encourage work that is:

• Aspirationally casual: measuring outcomes causally but not always through randomized control trials.

• The goal is not to shut down algorithms to to make positive contributions that generat solutions.

This is a difficult problem because:

• Lack of variation in accountability, enforcements, and interventions.

• It’s unclear what outcomes should be measure and how. This is context-dependent

• It’s unclear which interventions are the highest priority

Why progress is possible: There’s a lot of good activity in this space. And it’s early in the topic so there’s an ability to significantly influence the field.

What are the barriers for success?

• Incomplete understanding of contexts. So, think it in terms of socio-cultural approaches, and make it interdisciplinary.

• The topic lies between disciplines. So, develop a common language.

• High-level triangulation is difficult. Examine the issues at multiple scales, multiple levels of abstraction. Where you assess accountability may vary depending on what level/aspect you’re looking at.

Handling Uncertainty

The problem: How might we holistically treat and attribute uncertainty through data analysis and decisions systems. Uncertainty exists everywhere in these systems, so we need to consider how it moves through a system. This runs from choosing data sources to presenting results to decision-makers and people impacted by these results, and beyond that its incorporation into risk analysis and contingency planning. It’s always good to know where the uncertainty is coming from so you can address it.

Why difficult:

• Uncertainty arises from many places

• Recognizing and addressing uncertainties is a cyclical process

• End users are bad at evaluating uncertain info and incorporating uncertainty in their thinking.

• Many existing solutions are too computationally expensive to run on large data sets

Progress is possible:

• We have sampling-based solutions that provide a framework.

• Some app communities are recognizing that ignoring uncertainty is reducing the quality of their work

How to evaluate and recognize success?

• A/B testing can show that decision making is better after incorporating uncertainty into analysis

• Statistical/mathematical analysis

Barriers to success

• Cognition: Train users.

• It may be difficult to break this problem into small pieces and solve them individually

• Gaps in theory: many of the problems cannot currently be solved algorithmically.

The presentation ends with a note: “In some cases, uncertainty is a useful tool.” E.g., it can make the system harder to game.

Adversaries, workarounds, and feedback loops

Adversarial examples: add a perturbation to a sample and it disrupts the classification. An adversary tries to find those perturbations to wreck your model. Sometimes this is used not to hack the system so much as to prevent the system from, for example, recognizing your face during a protest.

Feedback loops: A recidivism prediction system says you’re likely to commit further crimes, which sends you to prison, which increases the likelihood that you’ll commit further crimes.

What is the problem: How should a trustworthy algorithm account for adversaries, workarounds, and feedback loops?

Who are the stakeholders?

System designers, users, non-users, and perhaps adversaries.

Why is this a difficult problem?

• It’s hard to define the boundaries of the system

• From whose vantage point do we define adversarial behavior, workarounds, and feedback loops.

Unsolved problems

• How do we reason about the incentives users and non-users have when interacting with systems in unintended ways.

• How do we think about oversight and revision in algorithms with respect to feedback mechanisms

• How do we monitor changes, assess anomalies, and implement safeguards?

• How do we account for stakeholders while preserving rights?

How to recognize progress?

• Mathematical model of how people use the system

• Define goals

• Find stable metrics and monitor them closely

• Proximal metrics. Causality?

• Establish methodologies and see them used

• See a taxonomy of adversarial behavior used in practice

Likely approaches

• Security methodology to anticipating and unintended behaviors and adversarial interactions’. Monitor and measure

• Record and taxonomize adversarial behavior in different domains

• Test . Try to break things.

Barriers

• Hard to anticipate unanticipated behavior

• Hard to define the problem in particular cases.

• Goodhardt’s Law

• Systems are born brittle

• What constitutes adversarial behavior vs. a workaround is subjective.

• Dynamic problem

Algorithms and trust

How do you define and operationalize trust.

The problem: What are the processes through which different stakeholders come to trust an algorithm?

Multiple processes lead to trust.

• Procedural vs. substantive trust: are you looking at the weights of the algorithms (e.g.), or what were the steps to get you there?

• Social vs personal: did you see the algorithm at work, or are you relying on peers?

These pathways are not necessarily predictive of each other.

Stakeholders build truth through multiple lenses and priorities

• the builders of the algorithms

• the people who are affected

• those who oversee the outcomes

Mini case study: a child services agency that does not want to be identified. [All of the following is 100% subject to my injection of errors.]

• The agency uses a predictive algorithm. The stakeholders range from the children needing a family, to NYers as a whole. The agency knew what into the model. “We didn’t buy our algorithm from a black-box vendor.” They trusted the algorithm because they staffed a technical team who had credentials and had experience with ethics…and who they trusted intuitively as good people. Few of these are the quantitative metrics that devs spend their time on. Note that FAT (fairness, accountability, transparency) metrics were not what led to trust.

Temporality:

• Processes that build trust happen over time.

• Trust can change or maybe be repaired over time. “

• The timescales to build social trust are outside the scope of traditional experiments,” although you can perhaps find natural experiments.

Barriers:

• Assumption of reducibility or transfer from subcomponents

• Access to internal stakeholders for interviews and process understanding

• Some elements are very long term

What’s next for this workshop

We generated a lot of scribbles, post-it notes, flip charts, Slack conversations, slide decks, etc. They’re going to put together a whitepaper that goes through the major issues, organizing them, and tries to capture the complexity while helping to make sense of it.

There are weak or no incentives to set appropriate levels of trust

Key takeways:

• Trust is irreducible to FAT metrics alone

• Trust is built over time and should be defined in terms of the temporal process

• Isolating the algorithm as an instantiation misses the socio-technical factors in trust.

Be the first to comment »

I’m at a two-day inter-disciplinary workshop on “Trustworthy Algorithmic Decision-Making” put on by the National Science Foundation and Michigan State University. The 2-page whitepapers
from the participants are online. (Here’s mine.) I may do some live-blogging of the workshops.

Goals:

– Key problems and critical qustionos?

– What to tell pol;icy-makers and others about the impact of these systems?

– Product approaches?

– What ideas, people, training, infrastructure are needed for these approaches?

Excellent diversity of backgrounds: CS, policy, law, library science, a philosopher, more. Good diversity in gender and race. As the least qualified person here, I’m greatly looking forward to the conversations.

Be the first to comment »