Mapping and collecting census data on five American Indian reservations

‍Tribal data collection challenges

Household data collection is never cut-and-dry, especially on rural Native American Reservations, which often lack a complete inventory of households, consistent addressing systems, or even marked roads. Because these difficulties lead to lack of accurate records about houses in rural areas, rural populations are often undercounted compared to urban areas. It is quite common on many Native American Reservations for Tribal members to locate a house or mobile home on their own land without registering it with their Tribe’s housing authority, making it virtually impossible to find using existing records.

Surveying the Pine Ridge Reservation

In 2000, Village Earth began working on a 5-year longitudinal household survey on the Pine Ridge Reservation, which was funded by the National Science Foundation. For the survey to be statistically representative of the entire population of the Reservation, a complete list of households was required from which to draw a random sample. Since there were no complete household lists available, the team decided to use GIS and GPS technology to generate a list of coordinates for each house. Once the houses were located, they were plotted on reference maps, which were used by field crews when they were conducting interviews. Although this process of building and printing map atlases for the field crews was both time consuming and expensive, existing map-based digital data collection solutions were cost prohibitive at the time.

Proven methodology

The data collected during the first few years of the household survey was of such high quality that the tribes were able to use it to challenge the 2000 Federal Census Numbers used by US Department of Housing and Urban Development (HUD). The Census challenge was accepted by HUD in 2005 and resulted in the following changes:

• 81% increase in the HUD recognized population (from 15,861 to 28,787)‏
• 31% increase in the number of (American Indian/Alaska Native) AIAN households with less than 30% median family income
• 62% increase in the number of AIAN households between 30% and 50% median family income
• 137% increase with more than 1 person per room or without kitchen or plumbing
• $1,292,000 increase in Pine Ridge’s IHBG Allocation starting in 2006

A year later, Village Earth was asked by the neighboring Rosebud Reservation to help them challenge their census numbers. The same methods were used as on the Pine Ridge Reservation, but instead of using graduate students to collect the data, locals were hired and trained to conduct over 1200 interviews using paper surveys. Again, the data was accepted by HUD and resulted in the following changes:

• 230% increase in the number of AIAN households with household expenses greater than 50% of income
• 42% increase in the number of AIAN households with less than 30% median family income
• 34% increase in the number of AIAN households with less than 80% of median family income
• $713,150 increase in IHBG Allocation in 2007

Technical concerns

From a methodological perspective, this system worked well to ensure the integrity of the random sample; namely that we had a complete list of households from which to generate a sample and that field crews only interviewed houses from that sample. The technical approach, however, left much to be desired in terms of efficiency and team management. Major issues included:

Quality control monitoring. Since surveys were returned at the end of each week, discovering and fixing issues — such as an incomplete survey or unclear handwriting — was extremely difficult.

Distribution of work. It was difficult to ensure that that workers always had enough surveys so data collection wouldn’t be interrupted. High performers would constantly run out while low performers would be sitting on a stack of surveys.

Data digitization. The transcription of each survey into a database was time-consuming, costly, and error-prone.

Digital data collection with Fulcrum

In 2012 the the new Federal Census numbers were going to be used to reset Tribal allocations across the country. Five Tribal Housing Authorities from North Dakota and South Dakota, including Pine Ridge, Rosebud, Turtle Mountain, Cheyenne River, and Lower Brule were concerned about the impact that the new Census numbers would have on their Federal allocations. In a proactive response, they formed a joint-venture to collaborate on a needs assessment that would take place simultaneously on each of their Reservations.

With their history of past successes and an innovative proposal, Village Earth was awarded the contract to conduct the household mapping and needs assessment. This time, instead of using map books, handheld GPS, and paper surveys, Village Earth proposed to take advantage of the latest in tablet-based survey technology. After doing an exhaustive comparison of the different technologies available, taking into consideration available features, cost and technical support, the team decided to go with Fulcrum running on Android devices. Some of the basic features required included:

• Preload of the survey with the locations of households mapped using aerial photos. This capability allowed staff to first create maps using aerial photos, then to use Fulcrum to locate and verify them in person from the field. When it came to the survey phase, the locations of the households that were selected in the random sample could be loaded.
• The ability for field crews to track their location and the location of the pre-loaded households using offline basemaps. This feature eliminated the need for the map books.
• Creation of complex forms to perform data collection offline, and then periodically sync this data to a central server. As WiFi and cellular data can be spotty on rural reservations even with 4G-enabled tablets, teams didn’t want to have to rely on a connection for data collection.

Fulcrum also had additional features that made the App even more indispensable for the project. These included:

• Color-coded map markers – These were used to mark each household location based on the status of a particular survey field. For the household mapping phase, this allowed field crews to record the status of each household during the mapping verification phase (e.g. occupied, vacant, not-a-house). During the survey phase, this feature was used to record the number of attempts made at each household. HUD requires that you make at least 5 attempts before assigning a house non-response designation. Assigning each house a unique color made it easy for field crews to know which houses had not been visited, the ones that were complete, and the ones that required another attempt. As managers of the process, it allowed for tracking in real-time the progress of surveys.
• Record assignment – This feature allowed for more efficient management of the 50+ member field crew, distributed over two states and five Native American Reservations. A key challenge with field survey is work assignment, and making sure that high-performing staff always have enough to do. Because Fulcrum allows managers to assign specific records to specific people and monitor their progress in real-time, records could easily be reassigned to another person if needed, removing the needless interruptions in work.
• Parent-child relationships for individual surveys – This was important to meet HUD’s requirement to collect data on each member of each household. For example, data would be collected for the household in general, but with “child” records beneath to represent each member of the household.
• Layers to basemaps – This made it easy to assign work “zones” to specific members of the field crews. Polygons were created around specific geographies that would then be assigned to individuals. This feature also helped to isolate randomly selected geographies where site coordinators could perform quality control.
• Web-based management interface – This made it simple to manage 5 separate data collection forms, 4 site-coordinators and a 50+ member field crew. New crew members were added, forms updated on the fly, records assigned as needed, data downloaded to monitor quality, and basemaps built and uploaded, all in real-time.

Conclusions

The most impressive thing about Fulcrum was the support we received from the staff at Fulcrum. Whether it was a general question about how to use the app, an issue encountered, or a feature request, the staff were always available and eager to help.

Overall, I am convinced that digital data collection using Fulcrum produced higher-quality data at a much lower cost than any other available technology, especially when compared to traditional paper surveys. Features added since we conducted our survey, such as parent-child records (one-to-many relationships), and webhook notifications make me even more enthusiastic about using Fulcrum for our next data collection project.

If you are interested in learning more about Village Earth and the data collection support services we offer, please contact David Bartecchi at david@villageearth.org or by phone at 970-237-3002 ext. 504.

‍