It looks like TSA is trying to prevent an open and honest audit of their
activities.
---
Secure Flight Working Group Report
Since January, I have been a member of the Secure Flight Working Group,
evaluating the security and privacy of the program. Last month we
released our report.
Honestly, I didn't do any of the writing. I had given up on the
process, sick of not being able to get any answers out of TSA, and
believed that the report would end up in somebody's desk drawer, never
to be seen again. I was stunned when I learned that the ASAC made the
report public.
There's a lot of stuff in the report, but I'd like to quote the section
that outlines the basic questions that the TSA was unable to answer:
"The SFWG found that TSA has failed to answer certain key questions
about Secure Flight: First and foremost, TSA has not articulated what
the specific goals of Secure Flight are. Based on the limited test
results presented to us, we cannot assess whether even the general goal
of evaluating passengers for the risk they represent to aviation
security is a realistic or feasible one or how TSA proposes to achieve
it. We do not know how much or what kind of personal information the
system will collect or how data from various sources will flow through
the system.
"Until TSA answers these questions, it is impossible to evaluate the
potential privacy or security impact of the program, including:
"* Minimizing false positives and dealing with them when they occur.
* Misuse of information in the system.
* Inappropriate or illegal access by persons with and without permissions.
* Preventing use of the system and information processed through it for
purposes other than airline passenger screening.
"The following broadly defined questions represent the critical issues
we believe TSA must address before we or any other advisory body can
effectively evaluate the privacy and security impact of Secure Flight on
the public.
"*What is the goal or goals of Secure Flight? The TSA is under a
Congressional mandate to match domestic airline passenger lists against
the consolidated terrorist watch list. TSA has failed to specify with
consistency whether watch list matching is the only goal of Secure
Flight at this stage. The Secure Flight Capabilities and Testing
Overview, dated February 9, 2005 (a non-public document given to the
SFWG), states in the Appendix that the program is not looking for
unknown terrorists and has no intention of doing so. On June 29, 2005,
Justin Oberman (Assistant Administrator, Secure Flight/Registered
Traveler) testified to a Congressional committee that "Another goal
proposed for Secure Flight is its use to establish "Mechanisms
for...violent criminal data vetting." Finally, TSA has never been
forthcoming about whether it has an additional, implicit goal the
tracking of terrorism suspects (whose presence on the terrorist watch
list does not necessarily signify intention to commit violence on a flight).
"While the problem of failing to establish clear goals for Secure Flight
at a given point in time may arise from not recognizing the difference
between program definition and program evolution, it is clearly an issue
the TSA must address if Secure Flight is to proceed.
"What is the architecture of the Secure Flight system? The Working Group
received limited information about the technical architecture of Secure
Flight and none about how software and hardware choices were made. We
know very little about how data will be collected, transferred,
analyzed, stored or deleted. Although we are charged with evaluating the
privacy and security of the system, we saw no statements of privacy
policies and procedures other than Privacy Act notices published in the
Federal Register for Secure Flight testing. No data management plan
either for the test phase or the program as implemented was provided or
discussed.
"Will Secure Flight be linked to other TSA applications? Linkage with
other screening programs (such as Registered Traveler, Transportation
Worker Identification and Credentialing (TWIC), and Customs and Border
Patrol systems like U.S.-VISIT) that may operate on the same platform as
Secure Flight is another aspect of the architecture and security
question. Unanswered questions remain about how Secure Flight will
interact with other vetting programs operating on the same platform; how
it will ensure that its policies on data collection, use and retention
will be implemented and enforced on a platform that also operates
programs with significantly different policies in these areas; and how
it will interact with the vetting of passengers on international flights?
"How will commercial data sources be used? One of the most controversial
elements of Secure Flight has been the possible uses of commercial data.
TSA has never clearly defined two threshold issues: what it means by
"commercial data;" and how it might use commercial data sources in the
implementation of Secure Flight. TSA has never clearly distinguished
among various possible uses of commercial data, which all have different
implications.
"Possible uses of commercial data sometimes described by TSA include:
(1) identity verification or authentication; (2) reducing false
positives by augmenting passenger records indicating a possible match
with data that could help distinguish an innocent passenger from someone
on a watch list; (3) reducing false negatives by augmenting all
passenger records with data that could suggest a match that would
otherwise have been missed; (4) identifying sleepers, which itself
includes: (a) identifying false identities; and (b) identifying
behaviors indicative of terrorist activity. A fifth possibility has not
been discussed by TSA: using commercial data to augment watch list
entries to improve their fidelity. Assuming that identity verification
is part of Secure Flight, what are the consequences if an identity
cannot be verified with a certain level of assurance?
"It is important to note that TSA never presented the SFWG with the
results of its commercial data tests. Until these test results are
available and have been independently analyzed, commercial data should
not be utilized in the Secure Flight program.
"*Which matching algorithms work best? TSA never presented the SFWG with
test results showing the effectiveness of algorithms used to match
passenger names to a watch list. One goal of bringing watch list
matching inside the government was to ensure that the best available
matching technology was used uniformly. The SFWG saw no evidence that
TSA compared different products and competing solutions. As a threshold
matter, TSA did not describe to the SFWG its criteria for determining
how the optimal matching solution would be determined. There are obvious
and probably not-so-obvious tradeoffs between false positives and false
negatives, but TSA did not explain how it reconciled these concerns.
"What is the oversight structure and policy for Secure Flight?</b> TSA
has not produced a comprehensive policy document for Secure Flight that
defines oversight or governance responsibilities."
The members of the working group, and the signatories to the report, are
Martin Abrams, Linda Ackerman, James Dempsey, Edward Felten, Daniel
Gallington, Lauren Gelman, Steven Lilenthal, Anna Slomovic, and myself.
There's one more bizarre twist to this story. Near the end of the
process, the TSA hired someone named Larry Ponemon to assist us in
writing our report. He had two jobs: one was to edit what we had to
say, and the other was to herd the members of the working group into
actually writing something coherent. But it turned out that the TSA
gave him another, secret, task: to write a document verifying our work.
So on the one hand, he was our scribe and project leader, but he was
also a TSA spy.
I think this is unethical, although it's pretty clear that Ponemon was
duped by the TSA. (Ponemon defended himself to us by saying that that
he did not believe his report would be made public. He refused to say
anything in public about this, because -- I assume -- he wants future
work from the TSA.)
His report basically says that TSA is doing everything fine, but that
the documentation simply wasn't available to us when we wrote our
report. This is wrong, and my guess is that Justin Oberman simply lied
to him convincingly. But the matter is now being taken up by the DHS's
Data Privacy and Integrity Advisory Committee.
Our report:
<http://www.tsa.gov/interweb/assetlibrary/SFWG_Report_September_19_2005_
Final_V_1_.4.pdf> or <http://tinyurl.com/ccyzj>
<http://www.epic.org/privacy/airtravel/sfwg_report_091905.pdf>
Ponemon's report:
<http://www.tsa.gov/interweb/assetlibrary/Ponemon_Institute_report_Final
_V_1_.4.pdf> or <http://tinyurl.com/9o6g7>
The U.S. Department of Justice Inspector General released a report last
month on Secure Flight, basically concluding that the costs were out of
control, and that the TSA didn't know how much the program would cost in
the future.
<http://www.usdoj.gov/oig/reports/FBI/a0534/final.pdf>
In case you think things have gotten better, there's a new story about
how the no-fly list cost a pilot his job:
<http://www.boston.com/news/local/massachusetts/articles/2005/09/22/no_f
ly_action_takes_pilots_job> or <http://tinyurl.com/864eu>
EPIC has received a bunch of documents about continued problems with
false positives on the no-fly list:
<http://www.epic.org/foia_notes/note8.html>
Here's an article about some of the horrible problems people who have
mistakenly found themselves on the no-fly list have had to endure.
<http://www.wired.com/news/privacy/0,1848,68973,00.html>
And another on what you can do if you find yourself on a list.
<http://www.wired.com/news/privacy/0,1848,68974,00.html>
And lastly, the TSA is currently not going to use commercial databases
in its initial roll-out of Secure Flight. I don't believe for a minute
that they're shelving plans to use commercial data permanently, but at
least they're delaying the process.
<http://beta.news.com.com/2061-10796_3-5878893.html>
My previous posts about Secure Flight, and my involvement in the working
group:
<http://www.schneier.com/blog/archives/2005/01/secure_flight_p.html>
<http://www.schneier.com/blog/archives/2005/01/tsas_secure_fli.html>
<http://www.schneier.com/blog/archives/2005/03/tsa_lied_about.html>
<http://www.schneier.com/blog/archives/2005/03/gaos_report_on.html>
<http://www.schneier.com/blog/archives/2005/07/secure_flight.html>
<http://www.schneier.com/blog/archives/2005/08/secure_flight_n.html>
.
|
