News From OSA - ASA Exam Edition - June, 2001

The item analysis of the 2001 ASA exam is now complete. We counted 302 of the Saturday exam returns before we called a halt. The three sets of one hundred showed very similar results. A big thank you is due to our chief tabulator George Morgan, his computerized consultant Keith Wilcox and our graphics designer Rob Spencer.

Our returns from the Sabbath observer exam were also counted, but separately, since Personnel scrambles question order between the regular and the Sabbath observer exams.

The chart looks complex, but it is easy to use. For question #1 on the Saturday exam, all of the results are reported, but the answer chosen by the most candidates is in "bold" print. The most common answer is not necessarily the correct answer, but often it is. If 97% of the test takers picked a particular answer and you did not, you probably have good reason to worry.

For some questions, more than one answer was highlighted because no one answer was chosen by a majority of test takers. Instead, two different answers were each somewhat popular. The best example of this is question #71. Answers A and C each were chosen by 47% of the candidates.

Appeals. OSA's letter instructing candidates to request the right to protest was modified by the instructions given on the pink sheet handed out by the examiners.

As per the City's instructions, candidates had seven days to request a chance to review the test at some day in the future. Any member who missed the seven-day deadline can still send in a request and hope for the best.

The actual protest, original and four copies, does not have to be submitted until after July 9th, the fifth Monday after the exam. It is expected that the City answer key will be published by that date. Protests have to be postmarked no later than 30 days thereafter. The real question is how you can protest at all without a copy of the exam on hand. It is hard, but there are hundreds of candidates who will make the effort.

If you did not keep your answer key, you can only await the score card in the mail.

If you did keep your answer sheet but did not (or do not) ask to review the exam, you will have to base the protests on memory. Most of us cannot do this, but some can.

If you kept your answer sheet and did ask to review the exam, our analysis plus the City's list of correct answers will enable you to focus on protesting. You will protest the questions (a) that you got wrong and (b) that many others got "wrong" as well. On #71 almost everybody chose either A or C, and in such cases there is good reason to review the question carefully. Many times, after protests, Personnel has agreed that there are two correct answers to a badly written question.

The exam itself will be undergoing a validation process this month. We expect that our brothers and sisters at Personnel/DCAS will be using their data to do a thorough item analysis and will be checking the exam over very carefully. Once completed, the official answer key will be issued.

OSA regrets that the City no longer allows candidates to take their exams home. We believe the design is to actively discourage protests. On the other hand, we are assured by Joan Doheny, OSA Chapter Chairperson from DCAS/Personnel that the protests submitted are, in fact, carefully considered by the professional staff of the Agency.

Conclusions. The 2001 exam was administered quite well compared to earlier years. One school did start late, but only 45 minutes late, and the others started on time. The weather was nice for those on line before the exam. By afternoon, some classrooms were too warm but others, with a breeze, were comfortable.

High school seats are too small for our members and the desks are a trial, so in the future, we will ask DCAS to seek accommodations in a college.

In Canarsie, (the site of the famous football game outside the windows last exam), there was a car radio (alarm?) that distracted some candidates for a bit but the area was quiet thereafter.

The exam was not easy. There were 18 questions where the agreement on the right answer topped 90%, but there were also 19 questions where there was no majority choice at all. Put another way, on average, most contestants got those 19 questions wrong. OSA will be urging DCAS to add points to the raw scores as we have in the past. Our tool, as you would expect, will be the item analysis you have in hand. In the past, DCAS has modified the raw scores somewhat, so we will keep our hopes up.

ASSOCIATE STAFF ANALYST EXAM

Item Analysis Saturday June 9, 2001 Exam Based On 302 Answer Sheets

Question

No.

No

Answer

Answer

"A"

Raw No.

Answer

"A"

Percent

Answer

"B"

Raw No.

Answer

"B"

Percent

Answer

"C"

Raw No.

Answer

"C"

Percent

Answer

"D"

Raw No.

Answer

"D"

Percent

Most Common Answer

1

 

30

10%

10

3%

182

60%

80

26%

C

2

 

255

84%

15

5%

17

6%

15

5%

A

3

 

144

48%

16

5%

123

41%

19

6%

A c

4

 

6

2%

48

16%

159

53%

89

29%

C

5

 

5

2%

3

1%

3

1%

291

96%

D

6

 

7

2%

269

89%

8

3%

18

6%

B

7

 

16

5%

223

74%

58

19%

5

2%

B

8

 

26

9%

228

75%

27

9%

21

7%

B

9

 

243

80%

10

3%

13

4%

36

12%

A

10

 

19

6%

24

8%

180

60%

79

26%

C

11

 

67

22%

223

74%

8

3%

4

1%

B

12

 

251

83%

20

7%

6

2%

25

8%

A

13

 

73

24%

69

23%

7

2%

153

51%

D

14

 

0

0%

19

6%

4

1%

279

92%

D

15

 

34

11%

14

5%

230

76%

24

8%

C

16

1

36

12%

8

3%

24

8%

233

77%

D

17

 

43

14%

51

17%

178

59%

30

10%

C

18

 

9

3%

248

82%

30

10%

15

5%

B

19

 

3

1%

75

25%

52

17%

172

57%

D

20

1

11

4%

39

13%

21

7%

230

76%

D

21

 

169

56%

30

10%

12

4%

91

30%

A

22

 

10

3%

274

91%

4

1%

14

5%

B

23

 

1

0%

6

2%

2

1%

293

97%

D

24

 

9

3%

276

91%

4

1%

13

4%

B

25

 

1

0%

292

97%

9

3%

0

0%

B

26

 

16

5%

19

6%

13

4%

253

84%

D

27

 

36

12%

6

2%

17

6%

243

80%

D

28

 

39

13%

89

29%

29

10%

145

48%

D

29

 

232

77%

29

10%

19

6%

22

7%

A

30

 

11

4%

199

66%

34

11%

58

19%

B

31

1

16

5%

4

1%

10

3%

271

90%

D

32

 

197

65%

14

5%

58

19%

33

11%

A

33

 

24

8%

70

23%

178

59%

30

10%

C

34

 

29

10%

87

29%

172

57%

14

5%

C

35

 

119

39%

118

39%

43

14%

22

7%

A/B

36

1

140

46%

79

26%

44

15%

38

13%

A

37

 

44

15%

191

63%

64

21%

3

1%

B

38

 

9

3%

74

25%

80

26%

139

46%

D

39

1

46

15%

69

23%

153

51%

33

11%

C

40

1

54

18%

149

49%

38

13%

60

20%

B

41

 

5

2%

280

93%

11

4%

6

2%

B

42

 

283

94%

16

5%

3

1%

0

0%

A

43

 

2

1%

7

2%

256

85%

37

12%

C

44

 

0

0%

204

68%

7

2%

91

30%

B

45

 

275

91%

20

7%

1

0%

6

2%

A

46

 

52

17%

16

5%

9

3%

225

75%

D

47

 

5

2%

4

1%

288

95%

5

2%

C

48

 

2

1%

4

1%

12

4%

284

94%

D

49

 

6

2%

13

4%

283

94%

0

0%

C

50

 

251

83%

26

9%

10

3%

15

5%

A

51

 

6

2%

293

97%

3

1%

0

0%

B

52

 

52

17%

232

77%

2

1%

16

5%

B

53

4

138

46%

99

33%

28

9%

33

11%

A b

54

 

94

31%

67

22%

32

11%

109

36%

D a

55

 

25

8%

133

44%

30

10%

114

38%

B d

56

2

116

38%

95

31%

39

13%

50

17%

A b

57

1

54

18%

20

7%

125

41%

102

34%

C d

58

 

52

17%

43

14%

11

4%

196

65%

D

59

 

149

49%

23

8%

34

11%

96

32%

A

60

1

16

5%

23

8%

241

80%

21

7%

C

61

 

36

12%

215

71%

14

5%

37

12%

B

62

 

12

4%

27

9%

214

71%

49

16%

C

63

 

28

9%

31

10%

38

13%

205

68%

D

64

 

22

7%

259

86%

11

4%

10

3%

B

65

 

190

63%

82

27%

21

7%

9

3%

A

66

 

19

6%

10

3%

241

80%

32

11%

C

67

 

8

3%

6

2%

278

92%

10

3%

C

68

1

43

14%

34

11%

77

25%

147

49%

D

69

 

33

11%

7

2%

21

7%

241

80%

D

70

 

18

6%

94

31%

185

61%

5

2%

C

71

 

142

47%

14

5%

143

47%

3

1%

C/A

72

1

223

74%

53

18%

17

6%

8

3%

A

73

 

15

5%

8

3%

272

90%

7

2%

C

74

 

262

87%

8

3%

22

7%

10

3%

A

75

 

20

7%

30

10%

49

16%

203

67%

D

76

 

24

8%

27

9%

220

73%

31

10%

C

77

 

26

9%

35

12%

228

75%

13

4%

C

78

 

93

31%

38

13%

124

41%

47

16%

C a

79

1

75

25%

122

40%

69

23%

35

12%

B

80

1

46

15%

79

26%

108

36%

68

23%

C

81

 

68

23%

139

46%

23

8%

72

24%

B

82

 

99

33%

22

7%

168

56%

13

4%

C

83

 

220

73%

11

4%

28

9%

43

14%

A

84

 

248

82%

20

7%

16

5%

18

6%

A

85

 

164

54%

42

14%

15

5%

81

27%

A

86

 

33

11%

229

76%

25

8%

15

5%

B

87

 

4

1%

9

3%

256

85%

33

11%

C

88

 

276

91%

8

3%

12

4%

6

2%

A

89

 

270

89%

26

9%

3

1%

3

1%

A

90

1

52

17%

92

30%

139

46%

18

6%

C

 

ASSOCIATE STAFF ANALYST EXAM

Item Analysis Friday June 8, 2001 Exam Sabbath Observers Based On 9 Answer Sheets

Question No.

Answer "A"

Answer "B"

Answer "C"

Answer "D"

Most Common Answer

1

9

     

A

2

9

     

A

3

 

4

4

1

B/C

4

1

 

7

1

C

5

8

1

   

A

6

3

2

4

 

C a

7

1

2

6

 

C

8

7

1

 

1

A

9

1

3

 

5

D

10

     

9

D

11

 

3

6

 

C

12

1

7

1

 

B

13

8

   

1

A

14

   

8

1

C

15

3

6

   

B

16

1

7

1

 

B

17

 

3

 

6

D

18

1

 

2

6

D

19

2

 

1

6

D

20

3

1

3

2

A/C

21

 

8

 

1

B

22

     

9

D

23

 

7

 

2

B

24

 

8

1

 

B

25

 

1

1

7

D

26

1

   

8

D

27

6

   

3

A

28

 

1

7

1

C

29

1

3

5

 

C

30

3

6

   

B

31

2

4

1

2

B

32

4

4

1

 

A/B

33

 

2

1

6

D

34

4

2

3

 

A c

35

1

5

3

 

B

36

3

2

 

4

D a

37

8

   

1

A

38

1

7

1

 

B

39

 

1

 

8

D

40

6

 

3

 

A

41

     

9

D

42

1

 

8

 

C

43

7

1

 

1

A

44

1

8

   

B

45

3

6

   

B

46

3

6

   

B

47

4

 

1

4

A/D

48

1

5

 

3

B

49

 

9

   

B

50

8

 

1

 

A

51

 

1

8

 

C

52

 

7

 

2

B

53

9

     

A

54

1

   

8

D

55

1

 

8

 

C

56

 

1

 

8

D

57

 

9

   

B

58

 

5

4

 

B c

59

4

1

2

2

A

60

2

3

2

2

B acd

61

1

2

 

6

D

62

5

   

4

A d

63

   

8

1

C

64

5

3

 

1

A

65

   

7

2

C

66

 

2

 

7

D

67

1

8

   

B

68

7

1

1

 

A

69

1

 

8

 

C

70

   

9

 

C

71

1

1

1

6

D

72

1

   

8

D

73

1

3

5

 

C

74

4

 

5

 

C a

75

5

3

1

 

A

76

   

9

 

C

77

8

   

1

A

78

   

3

6

D

79

1

 

8

 

C

80

 

3

5

1

C

81

4

1

3

1

A c

82

2

1

3

3

C/D

83

4

3

2

 

A b

84

 

8

 

1

B

85

   

8

1

C

86

7

2

   

A

87

8

1

   

A

88

5

1

1

2

A

89

 

9

   

B

90

   

9

 

C

top
top
top
top