Annual Citizen's Survey - One of the weeklies filed a story this week regarding the City Commission's discussion of our annual citzen's survey. I would say the story was biased to make the city look bad, but that would be an insult to bias. I encourage anyone that's interested to go to the city's website and view the actual discussion for yourself using the video streaming feature - the link is right there on the home page, sarasotagov.com. What actually happened was a fairly long discussion of the survey results, what one can and cannot conclude from them, what value they have, and how sometimes the results on certain items can be skewed by events that may or may not be city related. At no point did anyone belittle the citizens of Sarasota as the Pelican reported. The discussion was about how to get the most meaning from the results possible. In some cases, it was theorized that certain results (that didn't square with what we intuitively believed what people might say) could mean that we hadn't informed our citizenry well enough. If that were true, that is something we can fix, but it's a far cry from trying to "PR" our way out of a negative response, as was also stated. Frankly I found the report of the discussion more than a little slanted, probably to the point of irresponsible. But don't take my word for it, watch for yourself and draw your own conclusion.
posted by Michael McNees at 3:34 PM
5 Comments:
Dear Beamer,
My name is Debra Figueroa and I am the city staff member responsible for overseeing the Citizen Survey. I wanted to answer some of the questions you posed on the City Manager’s Blog.
The City chose the National Research Center (NRC) to administer the Citizen Survey because they are endorsed by the ICMA (International City/County Manager’s Association) and are a highly respected independent research firm. If you want to find out more information on NRC, please go to their website at http://n-r-c.com
You do not need to be concerned about the number of surveys returned and the resulting analysis. With the 606 surveys returned, the City of Sarasota can be 95% confident that the margin of error for the survey was only 4 percentage points above or below the results reported. This means that if we sent out the survey an infinite number of times, 95% of the time we would get the same results within the same 4% margin of error.
The typical sample size (surveys sent out) for a municipality using the National Citizen Survey is 1,200. With this sample size, the NCR expects 400 returned surveys. 400 surveys provide a confidence level of 95% with a margin of error of 5%. The City of Sarasota increased the sample size to 3,000 to decrease the margin of error in the survey results.
In addition, the 539 surveys that were sent to vacant or not found addresses were not mistakes. In order for a mail out survey to be considered valid, it has to be random. The City of Sarasota has a large seasonal population. We send out the surveys in July and at this time, our seasonal population is not present. If we were to pre-determine which households selected to send the survey, the survey would no longer be random and its results would be questionable. We could decide to survey the population during season. However, we will not be seeking reimbursement from the NRC due to our population patterns.
In response to the footnote references the City of Denver, it is just a typo. I will point it out to NRC. Thank you for catching that mistake.
The overall cost of the survey was $20,500 and included 3 closed-ended questions, 1 open-ended question, normative comparisons, geographic comparisons, demographic comparisons, and the option of Spanish translation. The only true staff time spent on the survey was mine, and it was limited to around 40 hours total. Mailing costs are included in the payment sent to NRC.
Dear Beamer,
The "Don't Know" responses are included in Appendix A of the Report of Results. They start on page 34.
I think it's easy to see there what you want to see. I see in this meeting, and others, a city council that is not in touch with it's people at all. Even in your post, you say that answers 'didn't square with what we intuitively believed what people might say'. What gall to decide you know what was meant or should have been the answers.
Do you also realize the gall that it takes to look at what the people of the city say when they are given a voice and then imply that either we didn't know what we were saying, that we're just flat wrong, that other constiuents (larger than the VOTERS? the PEOPLE? easy to see what constituency wins around here) have a voice too, or that we're responding to some other government body than them when talking about traffic or parks or whatever.
So instead of interpreting things from the only voice the people have been given other than the ballot box, especially when commissioners literally tell the people 'you can say your piece but it won't make any difference' during meetings, they choose to go the other way. Well, the ballot box will get set again soon. Personally I think it's going to be a massacre. Hopefully it will be in time to see some of this 'make Sarasota into West Palm' stuff stop.
Nearly all of those close to me in town think that the city is getting too ugly downtown and that it's creeping further and further out into the rest of the city. That it's absurd how much the city gives away to rich developers but doesn't do anything to attract real business to downtown or get work done on some affordable housing. But in no words have I heard a commissioner or city leader say anything approaching this. It gets lip service at best and when the next 'deal' gives away a street, a sidewalk, a few million dollars, covers a neighborhood in shadows forever and renames Sarasota 'Sarasota, brought to you by Developer XYZ', none of us will be surprised.
But at the end of the day Mr. McNees, Danny Bilyeu wants you out. And if he wants you out, I want you in. Nobody has been less receptive to his own district and more receptive to development, so there must be something you're doing right to get his backers, I mean him, against you.
Couldn't a random sample of addresses from a list of valid addresses been used? Why does including invalid addresses make the sample valid?
How was the endorsement of ICMA earned by NRC? Why was the survey sent out during July in the first place instead of when seasonal residents are here since you included their obviously vacant addresses. It almost seems a deliberate attempt to avoid hearing from seasonal residents while claiming they were included but unavailable. Many seasonal residents do not receive any local postal mail.
Matthew - I must not have explained what I meant very well. When I talked about data that "didn't square with what we intuitively believed what people might say" all I mean is that we might have an idea, before a question is asked, how people are going to answer. If the answer is significantly different from what we expected, then we have to try and figure out what the gap is. Our discussion was all about understanding what were valid conclusions from the data.
As for the "other government bodies" issue, I'll give an example. Years ago I ran the utility system for a county south of here, and in the county's annual survey we saw a dive in people's historic level of confidence in the water system, which was fairly new and of good quality. It just so happened that the largest city in that county had that year had a "scandle" involving water meters that received huge play in the press. We theorized that one had affected the other. Were we right? I can't say definitively. But the excercise is not to deflect criticism (if that were the case, why ask the questions in the first place? That is after all our idea.) The point is to make an intellectually honest effort to understand the data. Really and truly, our efforts the vast majority of the time are to figure out what is the best course of action, and to provide services in the best and most efficient way possible. I understand a certain amount of cynicism about that, but cynicism is really completely disruptive to an honest exchange of ideas.
Post a Comment
<< Home