Teach42

Don’t you hate when you spend a really long time typing up an amazing blog post that you’re really proud of and then you accidently click apple-w and the whole thing goes *poof*? Yeah, me too. Well, if it was worth typing once, it’s worth typing twice.

There’s a new article on eSchool News online that addresses an interesting topic. It’s titledNew Test Gauges ICT Literacy and it discusses a new test that supposedly assess Information and Communications Technology literacy. There’s no question that we do need to assess our ability to teach students, and we need to be teaching students the key components of information literacy, so the creation of a test to do just that makes perfect sense. However, I’m not sold on the idea that we are going to be able to create a test to quantify a students’ mastery of these skills. I guess I’m just a little skeptical about our ability to standardize an assessment of this sort.

Let’s begin by taking a look at the basic components of the test:

The exam presents the test-taker with a challenge and gives him or her the resources to investigate, Ewing said. The exam unfolds over two hours, during which time the test-taker is asked to carry out a series of 16 simulated tasks. For instance, the test-taker might be asked to perform an advanced search based on the need to find certain information. He or she might then be asked to use that information to put together a graph or compose an eMail message that summarizes the results of the research and draws subsequent conclusions from the data.

My first issue is that they are employing “simulated tasks”. It sounds like they are creating some sort of a fake search engine that is keyed to provide certain information to people supplying specific keywords or combinations of keywords. Simlulated tasks lead to simulated results. I remember when I applied for a job at a temp agency, they tested me in Microsoft Access. Even though I’d never used the program before, I got a 99% on the test. They weren’t actually testing my ability to use Microsoft Access, they were testing my prior knowledge of Microsoft applications and my ability to transfer that knowledge to a new program. If they had asked me to build a basic database with specific information, I probably would have failed horribly at the time.

In reality, there are hundreds of ways to solve a problem. Given a specific problem, some people might do a basic keyword search to find an answer. Other people might do a category search to find authoritive websites on the matter and proceed from there. Some people might try to find a message board or mailing list about the subject, where they will ask the community for information. Others may IM a friend, scroll through recent blog posts, use tools like Furl or del.icio.us, and so on. In the real world, often there is not a single ‘best’ solution or ‘right’ answer to the problems people encounter. If the simulated search engine is designed to only return sites based on specific keywords, then the only people who will pass that test are people who figure out what the test creators were looking for. If that’s true, then they are no longer testing information literacy, they are testing students’ ability to figure out test creators’ logic.

The same thing goes for reporting the information you find. There are so many options available to people, from Word to wikis, Powerpoint to video conferencing and everything in between. How do they quantify a students’ ability to present the information at hand? It’s all so subject, I have a hard time imagining how a test like this could work.

I don’t argue with the fact that there does need to be some way to assess information literacy. I know that there’s a University who is capturing video of every step a student takes to solve a problem so they can analyze it and assess it. However, I’m just not sure it’s going to be possible to standardize these kind of results in a meaningful way. Lori Roth, a librarian, said “What’s so marketable about the ICT Literary Assessment is the creativity in its design.” Marketable? I sincerely hope that the goal is not to create a test that’s marketable. The textbooks in this country are highly marketable too. Of course, their simply designed to appeal to the two biggest markets, California and Texas. My hope is that they create a test to be valid and relevant, not marketable.

In the article, they do point out that not everyone is thrilled with the test. However, I sincerely wish they’d used a better example for a contrary opinion.

Another aspect of the test that concerns Wilder is that it “had elements in it that seemed to be at least 10 years out of date,” he said. “You go through the test and see that [it addresses] Boolean searching. That is a library-land concept that had some currency 10 or 12 years ago. Modern search engines don’t do that at all. It makes me cringe that we are testing for those kinds of things that are so desperately out of date.”

Wilder is a dean of libraries at a major university. I would hope that he knows that most search engines actually DO accept boolean searches and that they are a perfectly valid way of introducing people to the concept. Teaching studetns about the different between AND and OR and transfering those concepts to any search is extremely valuable. Some search engines default to AND, others default to OR. It’s crucial to know the difference between them so you know what type of search you’re actually doing when using an engine. If the test requires students to use AND or OR regardless of the situation, well then that is obviously a problem. For example, I know that Google is by default an AND search. If they’re simulating a google search, then to require a student to use AND is ridiculous.

I guess the inherant problem with a test like this is that information literacy encompasses such a diverse set of skills. There’s more than one way to skin a cat. And who’s to say which way is best?

Comments RSS TrackBack Identifier URI