Statscan can’t afford for data access to play favourites

Former Chief Statistician Wayne Smith’s critique of Shared Services Canada may have some merit as this example illustrates:

There were some curious and intriguing details behind the headlines of Statistics Canada’s monthly employment report, as there always are. It’s a serious shame – and a serious problem – that almost no one could see them.

Again.

The national statistical agency’s website was out of commission since early Friday morning, before the 8:30 a.m. ET release of the February labour force survey. As of late afternoon, Statscan’s website remained dark; the details of one of the most important economic indicators of its monthly calendar were invisible to the Canadian public all day.

(By the way, the report showed that the Canadian economy added an estimated 15,000 net new jobs in February, a bit better than economists had expected, and the unemployment rate dropped to 6.6 per cent, matching an eight-year low.)

You might recall that something like this happened before, about eight months ago, when Statscan’s systems were down for more than seven hours on another jobs-report Friday. Not to mention the many, many occasions that Statscan’s website has fizzled out for much briefer periods shortly after the release of major economic indicators, during moments of peak traffic scrambling for the fresh data.

At the time of this writing, we don’t know what the problem was with Friday’s system failure. Neither Statscan nor Shared Services Canada, the agency that oversees e-mail, data and network services across the vast breadth of the federal public service, got back to us with an explanation. Certainly past snafus have been placed at the feet of Shared Services, the $1.9-billion brainchild of the previous Conservative government that was supposed to streamline Ottawa’s complex tangle of information technology, but has instead been blamed for everything from AWOL paycheques to RCMP systems failures.

The previous head of Statscan, Wayne Smith, resigned last September over Shared Services’ handling of Statscan’s information systems, which he said had not only become “disruptive, ineffective, slow and unaffordable,” but compromised the independence and confidentiality of the statistical agency’s data.

Now, I’m not here to point fingers. But the point is that these Statscan failures, while maybe not the same risk to public safety as the RCMP’s problems, are more than just a nuisance to the economists and journalists who wallow in these economic numbers.

The system problems, when they arise, create inequitable distribution of information that is relied on, and more to the point traded on, by financial markets. That’s a serious problem.

In the case of Friday’s jobs data, instead of every market participant being able to see the same data at the same time on the same website, each was left to his or her own devices (literally and figuratively). The lucky ones had access to Bloomberg data and news terminals, the expensive yet indispensable toys of professional trading operations, where at least the basic highlights of the report would have been fairly quickly disseminated. Others could have turned to media reports from the smattering of news organizations that attended Statscan’s pre-release lockup (in which reporters were given the release in advance but kept sequestered in a room, unable to communicate the information until the moment of the release time).

But if you were in need of the deeper statistical details below the surface of these quick-hit reports, good luck. Even the research departments of the big banks were scrambling, relying on friendly contacts at Statscan to e-mail to them whatever data they could.

All of which not only delayed the dissemination of this key economic data to the public and to financial markets, but also resulted in some very uneven distribution – in terms of both the timeliness and the amount of information that reached different sets of ears and eyes with an interest in the data.

And the employment data are very significant indeed to the bond and currency markets, especially now. It has become increasingly evident that the direction the Bank of Canada will take on interest rates hinges substantially on the evolution of the labour market. In its rate decision earlier this month, the central bank pointed specifically to “subdued growth in wages and hours” as key evidence of “persistent economic slack” in Canada.

And indeed, the February jobs report showed that despite the improvements in hiring and the unemployment rate, growth in wages and hours worked remained disappointing. Knowledge of this spoke volumes to any bond or currency trader placing bets on the timing of future Bank of Canada rate moves. And some traders had access to this information long before others.

That’s simply unacceptable.

As long as these technology problems persist, they undermine the integrity of an independent, impartial national statistics provider. Access to critical data can’t play favourites, even if it’s by accident.

Source: Statscan can’t afford for data access to play favourites – The Globe and Mail

New census technology under close watch as Statcan looks to the future

Looking forward to the results from the long-form census as they come out (short-form provides overview, long-form fills in many details). Makes sense to use existing government data to extent possible (CRA income data should be more reliable than self-reporting):

With just days to go before the very first release of data from the 2016 census, there is an unusual calm outside Marc Hamel’s Statistics Canada office.

A calm before the storm, perhaps.

After all, Wednesday’s release will be watched closely by federal officials, demographers and urban planners — all of whom use the data to help political leaders make myriad decisions that affect the daily lives of Canadians.

This time around, however, some of the keenest observers will be census director Hamel and his staff, watching to see if their new census data-collection methods are hitting their mark.

Statistics Canada has been quietly working on a plan for 2026 to eliminate the mandatory short-form census that goes to every household, instead using existing government databases to conduct a virtual count of the population. The plan, if successful, could mean millions in savings for federal coffers.

The closer the census numbers are to the tests being conducted by Hamel’s team, the more likely that multiple pages of the census questionnaire will be dropped during the next count in 2021, or replaced altogether one day in the future with an electronic count of the population.

This year, for instance, the agency cut two pages about income from the long-form questionnaire and replaced the questions with readily available and, arguably, more reliable Canada Revenue Agency data. Other questions, too, will eventually be replaced with information from existing administrative databases, making it easier to collect the details that comprise the census portrait.

Hamel said the challenge for his staff is to find a way to accurately reflect the Canadian population as it is at any point in time.

“The census as we run it now is very high quality, so anything that we would come up with in the future would have to be as high quality as it is today,” said Hamel.

One particular challenge for an electronic census: address information in various administrative files doesn’t always correspond to where people actually live, making it hard to be confident people are being counted in the right places.

And what about technology?

The majority of Canadians filled out their census questionnaires online, cutting down the time required to input data, and helping to speed up the release of information. Hamel said there might be other technological changes coming for future censuses, but it’s hard to predict what that might entail when census day rolls around again in 2021.

The question that guides planning for the next census and beyond is simple: will this work the same way next time?

“Four years in census terms — for me anyway — it’s short. It’s not a long time to prepare to make sure that we get it right. But at the same time, from a technological point of view, it’s fairly long,” Hamel said.

“It’s always a bit difficult to predict how technology will evolve in a short period of time and how that might have an impact on how the census might be rolled out.”

The questions on the census are also likely to change by 2021, with consultations starting this fall on what things Statistics Canada should and shouldn’t be measuring any more. One question likely to change is about sex and gender, which this year didn’t include a third option for transgender Canadians, Hamel noted.

“Society keeps evolving, so I think that from a census point of view, the census questions and questionnaire should be evolving with it.”

Source: New census technology under close watch as Statcan looks to the future | Toronto Star

Population Projections for Canada and its Regions, 2011 to 2036

sc-2036-vismin-002While I expect most of my readers will have seen the media reports on the latest population projections and will be familiar with the trends, here is the StatsCan summary of their findings.

One of the more striking findings is the likelihood that despite federal and provincial efforts to diversify where immigrants choose to settle and remain, relatively little change is seen from the current concentration in the major census metropolitan areas.

And as is currently the case, Montreal is likely to remain less diverse in terms of immigrants and visible minorities compared not only to Toronto and Vancouver but also Calgary and Winnipeg.

The other finding is the large increase in the number of second generation immigrants, where one in five is expected to be in 2036.

The full report is worth reading given the range of detailed information it provides even if, like all scenarios and projections, a note of caution is required.

And overall, given these trends, it is even more important to ensure that we get our immigration, citizenship and multiculturalism policies right to ensure our continued relative success in integrating newcomers and their children into Canadian society.

Immigrant and second-generation populations

  • Based on the projection scenarios used, immigrants would represent between 24.5% and 30.0% of Canada’s population in 2036, compared with 20.7% in 2011. These would be the highest proportions since 1871.
  • In 2036, between 55.7% and 57.9% of Canada’s immigrant population could have been born in Asia, up from 44.8% estimated in 2011, while between 15.4% and 17.8% could have been born in Europe, down from 31.6% in 2011.
  • The proportion of the second-generation population, i.e., non‑immigrants with at least one parent born abroad, within the total Canadian population would also increase. In 2036, nearly one in five people would be of second generation, compared with 17.5% in 2011.
  • Together, immigrants and second-generation individuals could represent nearly one person in two (between 44.2% and 49.7%) in 2036, up from 2011 (38.2%).

Languages

  • According to all scenarios used for these projections, the population whose mother tongue is neither English nor French would be up and could account for between 26.1% and 30.6% of Canada’s population in 2036, versus 20.0% in 2011.
  • As in 2011, immigrants would make up the majority—close to 70% in all scenarios—of the population whose mother tongue is neither English nor French. However, close to 40% of these other-mother-tongue immigrants would have adopted English or French as the language spoken most often at home, either alone or with other languages.

Visible minority status

  • According to the results of these projections, in 2036, among the working-age population (15 to 64 years), of special interest for the application of the Employment Equity Act, between 34.7% and 39.9% could belong to a visible minority group, compared with 19.6% in 2011.
  • In all the projection scenarios, South Asian would still be the main visible minority group in 2036, followed by the Chinese. However, the most rapidly growing groups would be the Arab, Filipino and West Asian groups, given that they represent a higher proportion in the immigrant population than in the population as a whole.

Religion

  • The proportion of people who report having no religion in the total population would continue to increase, and could represent between 28.2% and 34.6% in 2036 (compared with 24.0% in 2011). This proportion would be similar to Catholics (between 29.2% and 32.8% in 2036, down from 2011 [38.8%]). In 2036, Catholicism would remain the religion with the largest number of followers.
  • The number of people affiliated with non-Christian religions could almost double by 2036 and could represent between 13% and 16% of Canada’s population, compared with 9% in 2011. The Muslim, Hindu and Sikh faiths, which are over-represented among immigrants compared to their demographic weight in the population as a whole, would see the number of their followers grow more quickly, even if it would continue to represent a modest share of the total Canadian population.

Regional analysis

  • The results of the different scenarios show that in all provinces and territories, the number and the proportion of immigrants in the population would increase between 2011 and 2036.
  • Based on all the projection scenarios, the geographic distribution of immigrants among the various regions in 2036 would be similar to the estimate in 2011. The vast majority (between 91.7% and 93.4%) would continue to live in a census metropolitan area (CMA). The three primary areas of residence for immigrants would remain Toronto (between 33.6% and 39.1%), Montréal (between 13.9% and 14.6%) and Vancouver (between 12.4% and 13.1%).
  • According to all the scenarios for these projections, more than one in two people in 2036 would be an immigrant or the child of an immigrant in Toronto (between 77.0% and 81.4%), Vancouver (between 69.4% and 74.0%), Calgary (between 56.2% and 63.3%) and Abbotsford – Mission (between 52.5% and 57.4%). In 2011, the corresponding proportions were 74.1% in Toronto, 65.6% in Vancouver, 48.0% in Calgary and 49.7% in Abbotsford – Mission.
  • The results of the projections show that the proportion of the working-age population (aged 15 to 64) who belong to a visible minority group would increase in all areas of the country, in all the scenarios. This proportion would surpass 40% in Toronto, Vancouver, Edmonton, Calgary, Winnipeg and Abbotsford – Mission. It would remain lower in non-metropolitan areas.
  • The results of the projections indicate that religious diversity would be up in all areas considered by 2036. The increase would be more substantial in areas that were the most homogeneous in 2011, i.e., Quebec (excluding Montréal) and in the Atlantic provinces, primarily because of the rise in the proportion of people who reported having no religion.
  • The most religiously diverse areas in 2011 would remain as such in 2036. Among them, Toronto, Vancouver, Calgary and Edmonton, which had a large proportion of immigrants among their population in 2011, would continue to be diverse, in particular as a result of the increase in the proportion of persons reporting a non-Christian religion.

Source: Population Projections for Canada and its Regions, 2011 to 2036

Census still vulnerable to political meddling, says former chief

More on Statistics Canada and the proposed changes to make it more independent but according to Wayne Smith, the former Chief Statistician, not fully.

Unlikely that any government would give StatsCan full independence and I remember a lively Cabinet discussion on some aspects of the 2001 Census:

The federal government’s bid to protect Statistics Canada from political interference has a significant oversight that exposes the census to the possibility of government meddling, says Canada’s former chief statistician.

Wayne Smith, who resigned abruptly from the agency in September, said newly introduced legislation doesn’t change the parts of the Statistics Act that give cabinet control over the content of the questionnaire.

That leaves the census – used by governments to plan infrastructure and services – vulnerable to the sorts of changes the Conservatives imposed in 2011 by turning the long-form census into a voluntary survey, Smith said.

“That’s a major flaw in this bill,” he said. “The government brought this bill in because of the census, but it’s failing to deal with the census.”

Smith described the bill as a first step towards broadening the agency’s authority over how information on all types of subjects is collected, analyzed and disseminated, shifting that authority away from the minister.

Innovation Minister Navdeep Bains, the minister responsible for Statistics Canada, would retain the right to decide what information Statistics Canada collects, including directing the agency to collect data on emerging areas like renewable energy. The bill also gives cabinet the ability to make methodological changes, such as making mandatory surveys voluntary as the Conservatives did with the 2011 census.

Source: Census still vulnerable to political meddling, says former chief

Shared Services likely to become ‘money pit,’ says Canada’s former chief statistician who quit two weeks ago

In his words:

Canada’s former chief statistician, who publicly quit his job two weeks ago on principle to the cheers of hundreds of Statistics Canada employees, says Shared Services Canada is doomed to fail.

“There’s a really good chance Shared Services Canada will turn into a money pit,” Mr. Smith told The Hill Times after he resigned publicly as Canada’s chief statistician on Sept. 16.

Mr. Smith stepped down after fighting unsuccessfully to free Statistics Canada from Shared Services’ government IT department, which Mr. Smith said jeopardizes the number-crunching agency’s independence and integrity. As well, he said the model the government created for the government-wide IT management is doomed to fail.

…As a result of his departure, he told The Hill Times that he’s been iced-out of the deputy minister community, one he said he had never really been “in,” and, so far, said his claims have been dismissed by the government.

Mr. Smith said he thinks it’s a wrong-headed move to transfer Statistics Canada’s informatics infrastructure to Shared Services, but believes it’s the senior ranks of the bureaucracy that is pushing “extremely strongly,” in favour of the Shared Services model.

“This idea came out of the bureaucracy, the most senior bureaucracy is very committed to it, they don’t want to walk away from it. [The] Privy Council Office, Treasury Board Secretariat, and more people in the most senior ranks are strongly committed to this,” said Mr. Smith, adding that they were the ones to sell it to the current government as worth continuing.

“I think the government wants to believe it,” he said.

The $2-billion department, Shared Services, was created in 2011 by the previous Conservative government to consolidate and modernize the Government of Canada’s IT networks and personnel by 2020, a deadline it’s now uncertain about meeting, given extensive delays and potential greater costs than initially thought. Its three key tasks are to amalgamate all government email systems, merge data centres, and consolidate IT networks.

….Mr. Smith said, in principle, the job of the national statistics office is centered around information and technology, and “everything we do, from drawing samples, to collecting via the internet, to processing survey data and disseminating survey data, it absolutely requires informatics to run efficiently, and well and properly. … When the government created Shared Services Canada it took our away our authority to acquire informatics infrastructure, hardware, the servers, and the file servers we needed to do our job, and they gave that authority to SSC.”

Mr. Smith said even though he had the budget to purchase the informatics systems he wanted, the decision-making power had been taken away from him.

“Therefore they can stop me from disseminating data, from producing data, simply by withholding or failing to provide the informatics infrastructure—the computing power—to do it. And it doesn’t really matter, at the end of the day, if they do it out of malicious intent or whether they do it out of incompetence; the result is the same,” said Mr. Smith.

He said there was an “unacceptable level of risk” in its data centre infrastructure, which the two departments disagreed on where the actual drives would be located and who would have access. He said there was an inability for Shared Services to deliver the additional capacity required to move ahead with Statistics Canada’s plans to enhance its website to be more user-friendly. And he said there was, at the time he left, a “lineup” of policy departments at Statistics Canada’s door asking for new data as a result of the Liberal government’s emphasis in evidence-based decision-making.

“More money got spent, the results aren’t there, and this is simply because the decisions are outside the control of Statistics Canada now,” said Mr. Smith.

He added that although this was the state of affairs when he resigned, he’s optimistic that because of his outspoken critiques, “every effort will be made to make sure my predictions don’t come true.”

In response to the allegations last week, senior officials from Shared Services and the Treasury Board Secretariat held a technical briefing where Shared Services Canada chief Ron Parker dismissed Mr. Smith’s concerns.

Mr. Parker said that he and Mr. Smith last met at an April meeting and there were “no technical or operational issues” raised. Mr. Smith said this is utterly false.

“I was appalled … for him to contend that there was no issues is absolutely absurd,” Mr. Smith said, adding that he recalls at that meeting raising a “litany of concerns.”

Mr. Smith said he thinks the government shouldn’t go further down the enterprise-wide IT path until a business plan and accountability model have been established between Shared Services and all partner departments. He said the government should be skeptical about its ability to deliver on such a massive transformation, pointing to the Phoenix pay system debacle that’s disrupted or affected the pay for 82,000 public servants. The Phoenix pay system has cost the government more than $50-million to fix, and the $398-million Email Transformation Initiative to move all government email addresses to one your.email@canada.ca system is on hold and 18 months past when it was supposed to be fully implemented.

The complaints from Statistics Canada are not the first from a department who is unsatisfied with Shared Services work. A number of departments are unhappy about the service they’ve received and some other departments that deal with sensitive data have explored ways to opt out of the system. So far, Mr. Parker says the plans do not include any departments opting out of the agreement.

Despite this, Mr. Parker declared the benefits of the enterprise approach remain clear, and “the partners are part of that model and therefore there’s nothing, nothing in the plan that envisions opting-out.”

….The decision to resign came after months of trying to bring attention to his concerns, said Mr. Smith, who has been raising issues since before the current government was elected and after, in meetings with the minister responsible for Statistics Canada, Innovation Minister Navdeep Bains (Mississauga-Malton, Ont.), and Privy Council Clerk Michael Wernick.

He thought the government’s promise to enhance his agency’s independence would bring sea change to fix his problems with Shared Services. When it didn’t and the issues with Shared Services Canada began to dominate conversations with employees who were saying it affected their ability to do their job, he decided he needed to make it clear he was prepared to resign. After that didn’t move the needle, he submitted his resignation letter to Prime Minister Justin Trudeau (Papineau, Que.) on Aug. 3, enough time he thought for them to implement an independent appointment process for his successor. That didn’t happen and instead the government appointed Anil Arora, who was working as an assistant deputy minister at Health Canada.

“When I penned that letter I thought that the odds were overwhelmingly against it having any impact other than me winding up resigned, and my interpretation of the situation was correct,” said Mr. Smith, who didn’t hear anything from anyone in government from the point of submitting his resignation until Sept. 15, when letters came from both Mr. Wernick and Mr. Trudeau, accepting his resignation.

After the way government has handled his resignation, he thinks Canadians should be skeptical about their commitments to Statistics Canada.

While he doesn’t see himself as a whistleblower, since resigning he said he’s received a lot of encouragement from employees at the agency, who have sent him emails supporting his move and thanking him for standing up. He’s received support from the national statistics council, from provincial and territorial counterparts, and international support.

“Everybody sees the issue, and they’re all living the consequences,” Mr. Smith said.

Source: Shared Services likely to become ‘money pit,’ says Canada’s former chief statistician who quit two weeks ago – The Hill Times – The Hill Times

Statistics Canada eyes the end of the short-form census

Other countries do this and makes sense, both from cost and accuracy perspectives. But complex transition:

The mandatory long-form census returned this year, a decade after it was last seen.

If things go as planned, a decade from now the short-form census won’t be seen again.

Statistics Canada is working on a plan for the 2026 census that would eliminate the mandatory short-form census that goes to every household and instead use existing government databases to conduct a virtual count of the population.

The plan would save taxpayers millions of dollars and provide the same information used by governments to plan roads, hospitals, schools and other public services.

Documents obtained by The Canadian Press under the Access to Information Act paint a detailed picture of what officials hope to have in place by 2026: a digital register of every Canadian that could be updated every five years, if not annually, and a smaller long-form questionnaire.

“This approach to replace the short-form questionnaire will require a complete redesign of the long-form questionnaire,” reads the April report provided to former chief statistician Wayne Smith.

The agency said in a statement that it hasn’t yet determined its approach for the 2021 census, but made no direct reference to the 2026 count. The statement said the agency “conducts ongoing research activities to determine the most efficient way of collecting census information.”

Source: Statistics Canada eyes the end of the short-form census – Macleans.ca

Chief statistician resigns over government’s failure to ‘protect the independence’ of StatsCan

So both the Harper and Trudeau governments have lost a chief statistician on points of principle:

Canada’s chief statistician has resigned in protest over what he says is the federal governments’ failure to protect Statistics Canada’s independence.

Wayne Smith says the government’s decision to create Shared Services Canada and centralize all information technology services across government has compromised Statistics Canada’s ability to fulfil its mandate.

“I have made the best effort I can to have this situation remediated, but to no effect,” Smith said in a note to the National Statistical Council, which advises him. “I cannot lend my support to government initiatives that will purport to protect the independence of Statistics Canada when, in fact, that independence has never been more compromised,”

“I do not wish to preside over the decline of what is still, but cannot remain in these circumstances, a world-leading statistical office.”

Shared Services was created by the previous government to centralize and standardize information technology across the federal government in a bid to save money. It has struggled to meet expectations with several agencies, including the RCMP and the Canadian Forces, which have complained of data centre crashes, red tape, bad customer service and unpaid bills.

Smith said he had issued a warning that ever since Statistics Canada began relying on Shared Services for its IT, the research department had begun losing control of the information it collects from Canadians through operations such as the long-form census.

In the note, Smith argued that Shared Services holds “an effective veto over many of Statistics Canada’s decisions concerning the collection, processing, storage, analysis and dissemination of official statistics through denial or constructive denial of essential services.”

“Statistics Canada is increasingly hobbled in the delivery of its programs through disruptive, ineffective, slow and unaffordable supply of physical informatics services by Shared Services Canada,” he added.

Failure to convince government

Smith wrote in a separate note to staff that he tried to convince the Liberal government to correct the situation.

“I have not succeeded,” he wrote.

“I believe it is the professional duty of a national statistician to resign if the independence of the national statistical office — as envisioned in documents endorsed by Canada such as the United Nations Fundamental Principles of Official Statistics and the OECD Recommendation on Good Statistical Practice — is compromised.”

 ‘I think we do need to re-examine this whole approach to trying to centralize government services and cut costs.’– Erin Weir, NDP MP

In a statement issued by her office, Public Services Minister Judy Foote said the government “is committed to effective, efficient and secure service delivery to Canadians through modernizing government operations.”

Source: Chief statistician resigns over government’s failure to ‘protect the independence’ of StatsCan – Politics – CBC News

More detailed article with commentary by Kathryn May and quotes by former Chief Statistician Ivan Fellegi: Chief statistician butted heads with federal government over Shared Services Canada — and lost

Garbage in, garbage out: Canada’s big data problem

A reminder that despite the restoration of the Census, there still remain significant gaps in the collection, methodologies and dissemination of statistical data by the government:

In a recent article in the Toronto Star, Paul Wells lays out what he sees as Prime Minister Trudeau’s game plan for slowing Canada’s brain drain and making science pay. “Over the next year,” he writes, “the Trudeau government will seek to reinforce or shore up Canada’s advantage in three emerging fields: quantum tech, artificial intelligence and big data and analytics.”

As he should. If that’s the plan, it’s a good one. Canada’s future prosperity depends on our ability to innovate and retain the best talent in those three fields.

What we call “big data analytics” works by finding previously unknown patterns in the huge blocks of data that very large organizations — governments, for example — grow around themselves constantly, like coral. Finding those patterns can point the way to new efficiencies, new ways to fight crime and disease, new trends in business. But as with any complex system, what you get depends on what you put in. If the inputs aren’t accurate, the results won’t be, either. So before we embrace the “big data revolution”, we may want to look first at the worsening quality of the data our federal government produces, and that businesses, activists and social planners use.

Take something as basic as divorce. Statistics Canada first started reported marriage rates in 1921, divorce rates in 1972; it stopped collecting both data streams in 2011, citing “cost” concerns.

Marriage and divorce rates are exactly the kinds of data streams consumers of big data want collected, because they affect so many things: government policies, job markets, the service sector, housing starts — you name it. Having abandoned the field now for five years, StatsCan’s data volume on marital status isn’t nearly as useful as it might have been.

Take wildlife conservation. Recently an Ontario provincial backbencher proposed a private members bill to allow for unlimited hunting of cormorants. The bill’s proponent says the species is experiencing a population explosion. And we don’t know if he’s right or wrong — because the feds stopped collecting that data in 2011.

open quote 761b1bCanada used to publish statistical reports that were every bit as good as the Americans’ — in some cases, better. Then we stopped.

Here’s another big data blind spot: gasoline imports. After having reported data on gasoline imports regularly since 1973, StatsCan has been suppressing the numbers since 2013 due to what it calls “privacy” concerns. In the last reporting year, 2012, a staggering amount of imported gasoline came into the country — almost 4 billion litres.

Now, if you were thinking of expanding your oil refinery, or wanted to know more about how dependent this country is on foreign fuel, this would be pretty precious data — the kind you’d probably pay for. But the data aren’t reliable — any more than the StatsCan data on gasoline demand by province, which we use to work out whether carbon taxes are actually reducing demand for gasoline. It’s bad data; it has been for years. You’d think someone in the higher echelons of the federal or provincial governments would get annoyed.

Combing through StatsCan’s archive of reports can be a bewildering experience, even for experts. Its online database, CANSIM, is easy enough to use. It’s the reports themselves that sometimes fail you.

Say you want to understand trends in Ontario’s demand for natural gas. You’d start by looking at CANSIM table 129-0003, which shows an increase in sales of natural gas in 2007 over 2006 of 85 per cent. “Ah,” you think to yourself, “that must be because of the conversion of coal-burning plants to gas.” But no, that change occurred years later. Ask StatsCan and they’ll tell you that they changed their methodology that year — but didn’t bother re-stating the previous years’ numbers under the same methodology. Individually, the numbers are accurate — but the trend stops making sense.

StatsCan changed its methodology again this year; it now warns researchers to take care when comparing current and historical data. That’s an improvement over changing the methodology without telling anyone but it isn’t very helpful for understanding long-term trends.

And this isn’t just StatsCan’s problem. The National Energy Board published an excellent report showing where Canada’s crude ends up in the United States. Industry analysts use the numbers to understand the reasons why light and heavy crude are selling for what they’re selling for south of the border.

The NEB stopped reporting the data after September 2015. Ask why, and this is the response you get: “The Board has decided to discontinue publication of this data while we re-evaluate our statistical products.” That, of course, was a year ago.

Source: Garbage in, garbage out: Canada’s big data problem

Statscan fails to keep pace with seniors’ living arrangements

Valid concerns given the aging of the population:

In the wake of the 2016 census, researchers say they’re increasingly worried about limited data on a key segment of Canada’s booming senior population.
For the latest census distributed in May, Statistics Canada allowed administrators of nursing and retirement homes to complete a short-form census on behalf of residents. The agency also omitted the long-form census for all “collective dwellings,” which include hospitals, work camps and correctional institutions.

The move has irked some seniors and sparked calls from researchers for Statistics Canada to revise the 2021 census delivery as new models of senior living crop up.

“It’s something I’m concerned about with the aging of our population,” said Doug Norris, chief demographer at Environics Analytics and a census expert. “The data on our elderly population needs more attention than it’s gotten.”

Last year marked the first time that Canada had more people aged 65 and over – 16.1 per cent, or 5.8 million Canadians – than those 14 and under.

Without long-form results, Mr. Norris said researchers will lack crucial data about seniors’ income, ethnicity and education, among other findings. The data would be able to pinpoint demographic trends that have health implications, and shape myriad social policies, including seniors’ housing.

“Depending on the research and topic, it could be very important to include that group [of seniors], especially if you were doing anything health-related,” said Mr. Norris, who spent nearly 30 years at Statscan.

In the 2011 census, 378,000 people were counted in nursing and retirement homes classified as collective dwellings. This year, as in 2011, the short-form census – which captures age, gender, marital status and languages spoken – was distributed only to administrators of such homes.

Many questions on the long-form census, such as employment information, would not apply to seniors in nursing and retirement homes, said Geoff Bowlby, Statscan’s director-general of collection and regional services.

http://www.theglobeandmail.com/news/national/statscan-fails-to-keep-pace-with-seniors-living-arrangements/article31305272/ 

StatsCan looking for powers to make all surveys mandatory, compel data from companies

Will be interesting to see how the official opposition responds to this or whether, given the recognition by some prominent Conservatives that the arguments used to justify replacing the 2011 Census by the NHS were weak and wrong-headed, it lets this pass without comment:

Statistics Canada is privately floating the idea of new powers that would make all of its surveys mandatory by default and force certain companies to hand over requested data, such as credit card transactions and Internet search records.

Currently, the agency can ask for any information held by governments and businesses, but officials have long found it hard to get information like point-of-sale transactions that could give a more detailed and accurate picture of household spending.

The agency’s proposal would compel governments and companies to hand over information, and levy fines to discourage “unreasonable impositions” that “restrict or prevent the flow of information for statistical purposes.”

Corporate fines would depend on a company’s size and the length of any delays. The changes would also do away with the threat of jail time for anyone who refuses to fill out a mandatory survey, such as the long-form census.

The recommendations, contained in a discussion paper Statistics Canada provided to The Canadian Press, would enshrine in law the agency’s independence in deciding what data it needs and how to collect it.

New legislation to update the Statistics Act is expected to be tabled this fall, and the Liberals have promised to give Statistics Canada more freedom from government influence.

The current law permits the federal government to make unilateral changes — eliminating longitudinal studies about the Canadian population, for instance, or making the long-form census a voluntary survey, a Statcan spokesperson said.

Should the federal Liberals agree to the agency’s proposals, it would build a political wall between the government and Statistics Canada and ensure statistical decisions by the chief statistician take priority over political considerations.

StatsCan needs independence says Bains

Innovation Minister Navdeep Bains, who is responsible for Statistics Canada, said the government is still reviewing the Statistics Act. He said the government is committed to “strengthening the independence of Statistics Canada.”

“For a national statistical office to be credible, there must be a high degree of professional independence,” Bains said in a written statement.

“Canadians need to trust that their data are produced according to strict professional standards, ethics and scientific principles.”

Source: StatsCan looking for powers to make all surveys mandatory, compel data from companies – Politics – CBC News