Examining the Validity and Reliability of Data Collected by Online and Face-To-Face Survey Techniques
Yıl 2022,
, 397 - 410, 29.12.2022
Selçuk Efe Küçükkambak
,
Ece Armağan
Kaynakça
- Alreck, P., & Settle, R., 2004. Survey Research Handbook, Third edition. McGraw Hill, New York.
- Aquilino, W. S. (1991). Telephone Versus Face-To-Face Interviewing For Household Drug Use Surveys. International Journal of the Addictions, 27(1), 71-91.
- Aquilino, W. S. (1994). Interview Mode Effects In Surveys Of Drug And Alcohol Use: A Field Experiment. Public Opinion Quarterly, 58(2), 210-240.
- Bayart, C., & Bonnel, P., Morency, C., (2009). Survey Mode Integration And Data Fusion: Methods And Challenges, in “Transport Survey Methods: Keeping up with a Changing
World”. In: Bonnel, P., Lee Gosselin, M., Zmud, J., & Madre J.-L. (Ed.). Emerald Press, pp: 587-611.
- Bayart, C., & Bonnel, P. (2015). How To Combine Survey Media (Web, Telephone, Face-To-Face): Lyon And Rhône-Alps Case Study. Transportation Research Procedia, 11, 118-135.
- Bayart, C., & Bonnel, P., 2012. Combining Web and Face-to-face in Travel Surveys: Comparability Challenges. Transportation, 39(6), 1147-1171.
- Berger, J., T., & Halligan, R., M., (2012). Trademark Surveys, Oxford University Press Inc., New York.
- Biemer, P.P. & Lyberg, L.E. (2003). Introduction to Survey Quality. New York: John Wiley.
- Bonnel, P., 2003. Postal, Telephone And Face-To-Face Surveys: How Comparable Are They? in “Transport Survey Quality and Innovation”. In: Stopher, P.R., Jones, P.M. (Ed.). Pergamon, Oxford, pp. 215-237.
- Burns, A. C., Bush, R. F., & Sinha, N. (2017). Marketing Research, Eight (Global) Edition. Harlow: Pearson.
- Burns, A.C. & Bush, R.F. Marketing Research, 2nd ed. Upper Saddle River, NJ: Prentice Hall, 1998
- Byrne, M. B. (2016) Structural Equation Modeling with Amos Basic Concepts, Applications, and Programming, Third Edition, Routledge, New York.
- Christian, L.M., Dillman, D.A., & Smyth, J.D., 2008. The Effects Of Mode And Format On Answers To Scalar Questions In Telephone And Web Surveys. In: Lepkowski, J., Tucker, C.,
Brick, M., de Leeuw, E., Japec, L., Lavrakas, P., Link, M., & Sangster, R. (Eds.), Advances in Telephone Survey Methodology. Wiley-Interscience, New York, ss. 250–275 (Chapter 12).
- Couper, M. P. (2000). Web Surveys: A Review Of Issues And Approaches. The Public Opinion Quarterly, 64(4), 464-494.
- Couper, M. P., Traugott, M. W., & Lamias, M. J. (2001). Web Survey Design And Administration. Public Opinion Quarterly, 65(2), 230-253.
- Couper, M.P. & Nicholls, W.L. II (1998). The History and Development of Computerassisted Survey Information Collection Methods. In M.P. Couper, R.P. Baker, J. Bethlehem, C.Z.F.
Clark, J. Martin, W.L. Nicholls, II, & J.M. O’Reilly (eds), Computer-assisted Survey Information Collection New York: John Wiley.
- Couper, M.P., & Bosnjak, M., 2010. Internet surveys, in The Handbook of Survey Research. In: Marsden, P.V., & Wright, J.D. (Ed). Bingley, Emerald, UK, ss. 527-556.
- Cuneo, A. Z. (2004, November). Researchers Flail As Public Cuts The Cord. Advertising Age, 75 (46), 3.
- Czaja, R. & Blair, J. (2005) Designing Surveys: A Guide to Decisions and Procedures (second edition). Thousand Oaks: Sage, Pine Forge Press.
de Leeuw, E. & Collins, M. (1997). Data Collection Methods and Survey Quality: An Overview. In L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, & D. Trewin (eds) Survey Measurement and Process Quality. New York: John Wiley.
- De Leeuw, E. D. (2005). To Mix Or Not To Mix Data Collection Modes In Surveys. Journal Of Official Statistics, 21(5), 233-255.
- Diamond, S. S. (2000). Reference Guide On Survey Research. Reference Manual On Scientific Evidence, 221-228.
- Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., & Messer, B. L. (2009). Response Rate And Measurement Differences In Mixed-Mode Surveys Using Mail, Telephone, Interactive Voice Response (IVR) and The Internet. Social Science Research, 38(1), 1-18.
- Dillman, D.A., 2007. Mail and Internet Surveys. The Tailored Design Method. Wiley, New Jersey.
- Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005). Comparing Data From Online And Face-To-Face Surveys. International Journal of Market Research, 47(6), 615-639.
- Field, A. (2009). Discovering Statistics Using SPSS: Book Plus Code For E Version Of Text. SAGE Publications Limited.
- Fornell, C., & Larcker, D. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39-50.
- Fricker, S., Galesic, M., Tourangeau, R., & Ting, Y. (2005, Fall). An Experimental Comparison Of Web And Telephone Surveys. Public Opinion Quarterly, 69 (3), 370–392.
- Gelb, G. M., & Gelb, B. D. (2007). Internet Surveys For Trademark Litigation: Ready Or Not, Here They Come. Trademark Rep., 97, 1073.
- George, D., & Mallery, M. (2010). SPSS for Windows Step by Step: A Simple Guide and Reference, 17.0 update (10a ed.) Boston: Pearson
- Greenberg, D. (2000, July/August). Internet Economy Gives Rise To Real-Time Research. Quirk’s Marketing Research Review, 14 (7), 88–90.
- Groves, R. M. (1979). Actors And Questions In Telephone And Personal Interview Surveys. Public Opinion Quarterly, 43(2), 190-205.
- Groves, R.M., Fowler, F.J., Jr., Couper, M.P., Lepkowski, J.M., Singer, E., & Tourangeau, R. (2004). Survey Methodology. New York: John Wiley.
- Holbrook, A. L., Green, M. C., & Krosnick, J. A. (2003). Telephone Versus Face-To-Face Interviewing Of National Probability Samples With Long Questionnaires: Comparisons Of Respondent Satisficing And Social Desirability Response Bias. Public opinion quarterly, 67(1), 79-125.
- Hsu, J. W., & McFall, B. H. (2015). Mode Effects In Mixed-Mode Economic Surveys: Insights From A Randomized Experiment. Finance and Economics Discussion Series, 2015-008, Washington: Board of Governors of the Federal Reserve System.
- Hu, L. T., & Bentler, P. M. (1999). Cutoff Criteria For Fit Indexes In Covariance Structure Analysis: Conventional Criteria Versus New Alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55.
- Kagerbauer, M., Manz, W., & Zumkeller, D., (2012). Analysis of PAPI, CATI and CAWI Methods for a Multi-Day Household Travel Survey, in:“Transport Survey Methods: Best Practice for Decision Making”. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., & Carrasco, J.A. (Ed.). Emerald, Bingley, pp. 289-303.
- Krysan, M., Schuman, H., Scott, L.J., & Beatty, P., (1994). Response Rates And Response Content In Mail Versus Face-To-Face Surveys. Public Opinion Quarterly 58, 381–399.
- Lam, L. W. (2012). Impact of competitiveness on salespeople's commitment and performance. Journal of Business Research, 65(9), 1328-1334.
- Lugtig, P., Gerty, J. L. M., Lensvelt-Mulders, R. F., & Greven, A. (2011). Estimating Nonresponse Bias And Mode Effects In A Mixed-Mode Survey. International Journal of Market Research, 53 (5), 669–686.
- Macer, T., & Wilson, S. (2015, April). FocusVision Annual Market Research Technology Report, Meaning Ltd, London, ss. 17.
- Marta-Pedroso, C., Freitas, H., & Domingos, T. (2007). Testing For The Survey Mode Effect On Contingent Valuation Data Quality: A Case Study Of Web Based Versus In-Person Interviews. Ecological Economics, 62(3-4), 388-398.
- Mora, M. (2011). Understanding The Pros And Cons Of Mixedmode Research. Quirk’s Marketing Research Review, 25 (7), 50–54.
- Nathan, G. (2001). Telesurvey Methodologies For Household Surveys-A Review And Some Thoughts For The Future?. Survey Methodology, 27(1), 7-32.
- Nielsen, J. S. (2011). Use Of The Internet For Willingness-To-Pay Surveys: A Comparison Of Face-To-Face And Web-Based Interviews. Resource And Energy Economics, 33(1), 119-129.
- Özoğlu, B., & Bülbül, H. (2013). Güdülenmiş Tüketici Yenilikçiliği ve Algılanan Risk Ölçeklerinin Geçerlilik ve Güvenilirlik Çalışması. Journal of Alanya Faculty of Business/Alanya Isletme Fakültesi Dergisi, 5(3).
- See Hogg, A. (2002, July). Multi-Mode Research Dos And Don’ts. Quirk’s Marketing Research Review.
- Shawver, Z., Griffith, J. D., Adams, L. T., Evans, J. V., Benchoff, B., & Sargent, R. (2016). An Examination Of The WHOQOL-BREF Using Four Popular Data Collection Methods. Computers in Human Behavior, 55, 446-454.
- Stebbins, R. A. (2001). Exploratory Research İn The Social Sciences (Vol. 48). Sage.
- Szolnoki, G., & Hoffmann, D. (2013). Online, Face-To-Face And Telephone Surveys—Comparing Different Sampling Methods In Wine Consumer Research. Wine Economics and Policy, 2(2), 57-66.
- Tarnai, J., & Dillman, D.A., 1992. Questionnaire Context As A Source Of Response Differences In Mail Versus Telephone Surveys. In: Schwarz, N., Sudman, S. (Eds.), Context Effects in Social and Psychological Research. Springer-Verlag, New York.
- Thériault, M., Lee-Gosselin, M., Alexandre, L., Théberge, F., & Dieumegarde, L., (2012). Web versus Pencil and Paper Surveys of Weekly Mobility: Conviviality, Technical and Privacy Issues, in Transport Survey Methods: Best Practice for Decision Making. Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (Ed.). Emerald, Bingley, pp. 225-245.
- Thompson, B. (2004). Exploratory And Confirmatory Factor Analysis. American Psychological Association.
- Vandecasteele, B. & Geuens, M. (2010). Motivated Consumer Innovativeness: Concept, Measurement, And Validation, International Journal Of Research In Marketing, 27, 308-318.
- Wilson, P., Blackshaw, & N, Norris, P. (1988). An Evaluation Of Telephone İnterviewing On The British Labour Force Survey. Journal of Official Statistics, 4(4), 385.
- Windle, J., & Rolfe, J. (2011). Comparing Responses From İnternet And Paper-Based Collection Methods In More Complex Stated Preference Environmental Valuation Surveys. Economic Analysis and Policy, 41(1), 83-97.
Çevrimiçi ve Yüz Yüze Anket Teknikleri ile Toplanan Verilerin Geçerlilik ve Güvenilirliklerinin İncelenmesi
Yıl 2022,
, 397 - 410, 29.12.2022
Selçuk Efe Küçükkambak
,
Ece Armağan
Öz
İnternet ve bilgi teknolojileri alanında özellikle 2000’li yılların başından itibaren yaşanan gelişimler ve yenilikler pazarlama araştırmalarını da pek çok yönüyle etkilemiştir. Tüketici ve pazar hakkındaki bilgilere daha hızlı erişme ve yorumlama ihtiyacı; pazarlama araştırmalarında kullanılan yöntem ve tekniklerde birtakım yenilikleri de beraberinde getirmiştir. Bilgi ve teknoloji çağında yaşanan hızlı gelişmeler araştırmacıların veri toplama yöntemlerini yenilikçi ve çağdaş bakış açısıyla sorgulamasını, çağın gereklilikleriyle uyumlu bilimsel araştırma yöntemleri kullanmasını bir ihtiyaç haline getirmiştir. Bu çalışmada pazarlama alanındaki nicel araştırma yöntemlerinde araştırmacıların sıklıkla başvurduğu veri toplama tekniklerinden yüz yüze ve online anket yöntemleriyle toplanan verilerin geçerlilik ve güvenilirlikleri, örnek bir ölçme aracı kullanılarak karşılaştırmalı biçimde analiz edilmiştir. Araştırmada yüz yüze ve online olarak toplanan verilerin geçerlilik ve güvenilirliklerinin birbirine yakın olduğu, her iki veri grubunun birleştirilmesiyle elde edilen yeni veri grubunun da geçerli ve güvenilir olduğunu sonucuna ulaşılmıştır. Bu çalışmada ulaşılan sonuçların gelecekte pazarlama araştırmalarında yürütülecek çalışmalarda kullanılacak yenilikçi ve karma veri toplama yaklaşımları konusunda araştırmacılara yol göstermesi beklenmektedir.
Kaynakça
- Alreck, P., & Settle, R., 2004. Survey Research Handbook, Third edition. McGraw Hill, New York.
- Aquilino, W. S. (1991). Telephone Versus Face-To-Face Interviewing For Household Drug Use Surveys. International Journal of the Addictions, 27(1), 71-91.
- Aquilino, W. S. (1994). Interview Mode Effects In Surveys Of Drug And Alcohol Use: A Field Experiment. Public Opinion Quarterly, 58(2), 210-240.
- Bayart, C., & Bonnel, P., Morency, C., (2009). Survey Mode Integration And Data Fusion: Methods And Challenges, in “Transport Survey Methods: Keeping up with a Changing
World”. In: Bonnel, P., Lee Gosselin, M., Zmud, J., & Madre J.-L. (Ed.). Emerald Press, pp: 587-611.
- Bayart, C., & Bonnel, P. (2015). How To Combine Survey Media (Web, Telephone, Face-To-Face): Lyon And Rhône-Alps Case Study. Transportation Research Procedia, 11, 118-135.
- Bayart, C., & Bonnel, P., 2012. Combining Web and Face-to-face in Travel Surveys: Comparability Challenges. Transportation, 39(6), 1147-1171.
- Berger, J., T., & Halligan, R., M., (2012). Trademark Surveys, Oxford University Press Inc., New York.
- Biemer, P.P. & Lyberg, L.E. (2003). Introduction to Survey Quality. New York: John Wiley.
- Bonnel, P., 2003. Postal, Telephone And Face-To-Face Surveys: How Comparable Are They? in “Transport Survey Quality and Innovation”. In: Stopher, P.R., Jones, P.M. (Ed.). Pergamon, Oxford, pp. 215-237.
- Burns, A. C., Bush, R. F., & Sinha, N. (2017). Marketing Research, Eight (Global) Edition. Harlow: Pearson.
- Burns, A.C. & Bush, R.F. Marketing Research, 2nd ed. Upper Saddle River, NJ: Prentice Hall, 1998
- Byrne, M. B. (2016) Structural Equation Modeling with Amos Basic Concepts, Applications, and Programming, Third Edition, Routledge, New York.
- Christian, L.M., Dillman, D.A., & Smyth, J.D., 2008. The Effects Of Mode And Format On Answers To Scalar Questions In Telephone And Web Surveys. In: Lepkowski, J., Tucker, C.,
Brick, M., de Leeuw, E., Japec, L., Lavrakas, P., Link, M., & Sangster, R. (Eds.), Advances in Telephone Survey Methodology. Wiley-Interscience, New York, ss. 250–275 (Chapter 12).
- Couper, M. P. (2000). Web Surveys: A Review Of Issues And Approaches. The Public Opinion Quarterly, 64(4), 464-494.
- Couper, M. P., Traugott, M. W., & Lamias, M. J. (2001). Web Survey Design And Administration. Public Opinion Quarterly, 65(2), 230-253.
- Couper, M.P. & Nicholls, W.L. II (1998). The History and Development of Computerassisted Survey Information Collection Methods. In M.P. Couper, R.P. Baker, J. Bethlehem, C.Z.F.
Clark, J. Martin, W.L. Nicholls, II, & J.M. O’Reilly (eds), Computer-assisted Survey Information Collection New York: John Wiley.
- Couper, M.P., & Bosnjak, M., 2010. Internet surveys, in The Handbook of Survey Research. In: Marsden, P.V., & Wright, J.D. (Ed). Bingley, Emerald, UK, ss. 527-556.
- Cuneo, A. Z. (2004, November). Researchers Flail As Public Cuts The Cord. Advertising Age, 75 (46), 3.
- Czaja, R. & Blair, J. (2005) Designing Surveys: A Guide to Decisions and Procedures (second edition). Thousand Oaks: Sage, Pine Forge Press.
de Leeuw, E. & Collins, M. (1997). Data Collection Methods and Survey Quality: An Overview. In L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, & D. Trewin (eds) Survey Measurement and Process Quality. New York: John Wiley.
- De Leeuw, E. D. (2005). To Mix Or Not To Mix Data Collection Modes In Surveys. Journal Of Official Statistics, 21(5), 233-255.
- Diamond, S. S. (2000). Reference Guide On Survey Research. Reference Manual On Scientific Evidence, 221-228.
- Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., & Messer, B. L. (2009). Response Rate And Measurement Differences In Mixed-Mode Surveys Using Mail, Telephone, Interactive Voice Response (IVR) and The Internet. Social Science Research, 38(1), 1-18.
- Dillman, D.A., 2007. Mail and Internet Surveys. The Tailored Design Method. Wiley, New Jersey.
- Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005). Comparing Data From Online And Face-To-Face Surveys. International Journal of Market Research, 47(6), 615-639.
- Field, A. (2009). Discovering Statistics Using SPSS: Book Plus Code For E Version Of Text. SAGE Publications Limited.
- Fornell, C., & Larcker, D. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39-50.
- Fricker, S., Galesic, M., Tourangeau, R., & Ting, Y. (2005, Fall). An Experimental Comparison Of Web And Telephone Surveys. Public Opinion Quarterly, 69 (3), 370–392.
- Gelb, G. M., & Gelb, B. D. (2007). Internet Surveys For Trademark Litigation: Ready Or Not, Here They Come. Trademark Rep., 97, 1073.
- George, D., & Mallery, M. (2010). SPSS for Windows Step by Step: A Simple Guide and Reference, 17.0 update (10a ed.) Boston: Pearson
- Greenberg, D. (2000, July/August). Internet Economy Gives Rise To Real-Time Research. Quirk’s Marketing Research Review, 14 (7), 88–90.
- Groves, R. M. (1979). Actors And Questions In Telephone And Personal Interview Surveys. Public Opinion Quarterly, 43(2), 190-205.
- Groves, R.M., Fowler, F.J., Jr., Couper, M.P., Lepkowski, J.M., Singer, E., & Tourangeau, R. (2004). Survey Methodology. New York: John Wiley.
- Holbrook, A. L., Green, M. C., & Krosnick, J. A. (2003). Telephone Versus Face-To-Face Interviewing Of National Probability Samples With Long Questionnaires: Comparisons Of Respondent Satisficing And Social Desirability Response Bias. Public opinion quarterly, 67(1), 79-125.
- Hsu, J. W., & McFall, B. H. (2015). Mode Effects In Mixed-Mode Economic Surveys: Insights From A Randomized Experiment. Finance and Economics Discussion Series, 2015-008, Washington: Board of Governors of the Federal Reserve System.
- Hu, L. T., & Bentler, P. M. (1999). Cutoff Criteria For Fit Indexes In Covariance Structure Analysis: Conventional Criteria Versus New Alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55.
- Kagerbauer, M., Manz, W., & Zumkeller, D., (2012). Analysis of PAPI, CATI and CAWI Methods for a Multi-Day Household Travel Survey, in:“Transport Survey Methods: Best Practice for Decision Making”. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., & Carrasco, J.A. (Ed.). Emerald, Bingley, pp. 289-303.
- Krysan, M., Schuman, H., Scott, L.J., & Beatty, P., (1994). Response Rates And Response Content In Mail Versus Face-To-Face Surveys. Public Opinion Quarterly 58, 381–399.
- Lam, L. W. (2012). Impact of competitiveness on salespeople's commitment and performance. Journal of Business Research, 65(9), 1328-1334.
- Lugtig, P., Gerty, J. L. M., Lensvelt-Mulders, R. F., & Greven, A. (2011). Estimating Nonresponse Bias And Mode Effects In A Mixed-Mode Survey. International Journal of Market Research, 53 (5), 669–686.
- Macer, T., & Wilson, S. (2015, April). FocusVision Annual Market Research Technology Report, Meaning Ltd, London, ss. 17.
- Marta-Pedroso, C., Freitas, H., & Domingos, T. (2007). Testing For The Survey Mode Effect On Contingent Valuation Data Quality: A Case Study Of Web Based Versus In-Person Interviews. Ecological Economics, 62(3-4), 388-398.
- Mora, M. (2011). Understanding The Pros And Cons Of Mixedmode Research. Quirk’s Marketing Research Review, 25 (7), 50–54.
- Nathan, G. (2001). Telesurvey Methodologies For Household Surveys-A Review And Some Thoughts For The Future?. Survey Methodology, 27(1), 7-32.
- Nielsen, J. S. (2011). Use Of The Internet For Willingness-To-Pay Surveys: A Comparison Of Face-To-Face And Web-Based Interviews. Resource And Energy Economics, 33(1), 119-129.
- Özoğlu, B., & Bülbül, H. (2013). Güdülenmiş Tüketici Yenilikçiliği ve Algılanan Risk Ölçeklerinin Geçerlilik ve Güvenilirlik Çalışması. Journal of Alanya Faculty of Business/Alanya Isletme Fakültesi Dergisi, 5(3).
- See Hogg, A. (2002, July). Multi-Mode Research Dos And Don’ts. Quirk’s Marketing Research Review.
- Shawver, Z., Griffith, J. D., Adams, L. T., Evans, J. V., Benchoff, B., & Sargent, R. (2016). An Examination Of The WHOQOL-BREF Using Four Popular Data Collection Methods. Computers in Human Behavior, 55, 446-454.
- Stebbins, R. A. (2001). Exploratory Research İn The Social Sciences (Vol. 48). Sage.
- Szolnoki, G., & Hoffmann, D. (2013). Online, Face-To-Face And Telephone Surveys—Comparing Different Sampling Methods In Wine Consumer Research. Wine Economics and Policy, 2(2), 57-66.
- Tarnai, J., & Dillman, D.A., 1992. Questionnaire Context As A Source Of Response Differences In Mail Versus Telephone Surveys. In: Schwarz, N., Sudman, S. (Eds.), Context Effects in Social and Psychological Research. Springer-Verlag, New York.
- Thériault, M., Lee-Gosselin, M., Alexandre, L., Théberge, F., & Dieumegarde, L., (2012). Web versus Pencil and Paper Surveys of Weekly Mobility: Conviviality, Technical and Privacy Issues, in Transport Survey Methods: Best Practice for Decision Making. Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (Ed.). Emerald, Bingley, pp. 225-245.
- Thompson, B. (2004). Exploratory And Confirmatory Factor Analysis. American Psychological Association.
- Vandecasteele, B. & Geuens, M. (2010). Motivated Consumer Innovativeness: Concept, Measurement, And Validation, International Journal Of Research In Marketing, 27, 308-318.
- Wilson, P., Blackshaw, & N, Norris, P. (1988). An Evaluation Of Telephone İnterviewing On The British Labour Force Survey. Journal of Official Statistics, 4(4), 385.
- Windle, J., & Rolfe, J. (2011). Comparing Responses From İnternet And Paper-Based Collection Methods In More Complex Stated Preference Environmental Valuation Surveys. Economic Analysis and Policy, 41(1), 83-97.