Welcome login | signup
Language en es fr

Forum Post: Outsourced Intelligence: How the FBI and CIA Use Private Contractors to Monitor Social Media

Posted 6 years ago on June 13, 2013, 3:54 p.m. EST by LeoYo (5909)
This content is user submitted and not an official statement

Outsourced Intelligence: How the FBI and CIA Use Private Contractors to Monitor Social Media

Thursday, 13 June 2013 09:49 By Stephen Benavides, Truthout | News Analysis


Right now, companies like Palantir Technologies Inc, Booz Allen Hamilton, and i2 are mining your Facebook and Twitter data in an effort to discern whether you're a terrorist, have ties to terrorists or maybe just have the potential to someday become one. They also want to know if you have links to the Boston bombers, Kim Jong-un or Darth Vader, and they've been paid millions upon millions of dollars to do this on behalf of the US Special Operations Command, FBI, CIA, DIA (Defense Intelligence Agency), the Army, Marines and the Air Force. Initially a small start-up conveniently funded in part the CIA's nonprofit venture capital firm In-Q-Tel, Palantir Technologies is now the leading embodiment of online Big Brother.

Not only is Palantir Technologies not part of the US government, they'd like you think they're run by young, techy hipsters who are fond of stuffed Care Bears, My Little Ponies, hegdehogs and flip-flops, as the photos fronting their web site would indicate. In fact, this once "innocent" start-up - responsible for identifying the Chinese cyber espionage network Ghostnet - has become part and parcel to the intelligence failure leading up to the Boston and Watertown tragedies.

The software employed by the FBI and CIA is a "Java-based platform for analyzing, integrating, and visualizing several types of data," according to to Army Test and Evaluation Command (ETEC). It uses a provisioning database, where all modifications performed on database objects and entities are documented. The software allows for the plotting of current or historical targets of high-value individuals (HVI's) taking into account time, location, transfer of funds history, as well as social media and other online communications. While this may seem super technologically, according to the Operational Assessment Report approved by US Army Brigadier General Laura J. Richardson, the software was so easy to use, it took participants less than one hour of training to learn how to perform program functions allowing completion of assigned duties. The foundation of Palantir's software was initially developed as part of PayPal's fraud detection system, which taught computers to detect and flag suspicious money transfers so human analysts would be able to follow up. At issue was the software's inability to keep up with ever-changing tactics employed by criminals and the increasing volume of fraudulent transactions. In response, PayPal's computer scientists developed a system that would track the individual's computer, whom they did business with, where those people were located and fit it all into a transaction history database. Now instead of looking at a boring spreadsheet, analysts were able to visualize networks using streamlined graphical user interfaces (GUI), allowing them to see patterns that the previous software had missed. Palantir's government work remains classified, so we don't know exactly what changes were made to transform a fraud detection program into something that the FBI and CIA use to track high-value targets, but we can certainly take a guess that it is empowered by the Patriot Act.

While the classified nature of surveillance data transmitted to Palantir is obviously very powerful in tracking potential terrorists, actual terrorists and everything in-between, open source intelligence (OSINT) plays - at the minimum - a very close secondary role to the infamous warrantless wiretapping and other forms of surveillance employed by the government. OSINT is comprised of print, radio, television, and online-based media; social media-based community interactions such as Facebook and Twitter; public data contained in government budgets, reports, demographics, and hearings; academic literature; as well as information gained from geospatial software such as GIS. Communication services such as Twitter, Facebook and Gmail are all accessible through voluntarily designed backdoors. These backdoors allow direct access to databases and servers that hold all of your information, not to mention direct access to your personal profile or account.

If you thought that the FBI and CIA were the only ones watching you online, think again. Undersecretary of Homeland Security Charles Allen gave testimony before the House Homeland Security Committee's Intelligence, Information Sharing and Terrorism Risk Assessment Subcommittee that he had established a "Domestic Open Source Intelligence Enterprise" in support of the department's antiterrorism efforts. DHS is not the only one; there is also the Defense Intelligence Agency, National Geospatial Intelligence Agency, US Army Foreign Military Studies Offices, US Special Operations Command, US Strategic Command, INTERPOL, EUROPOL, Scotland Yard, the Mounties in Canada and every other self-respecting form of law enforcement agency.

When contacted about Palantir's stance on the use of open-source intelligence to track non-terrorists, and whether or not they considered this a privacy violation, Palantir failed to respond. There are any number of reasons that you might end up on the FBI or CIA's watch list, but primary reasons include having a criminal record for terrorist-related activity; actually knowing or communicating with known terrorists or terrorist organizations; and of course the catch-all: "material support" for terrorism. According to the Center for Constitutional Rights the material support statute, otherwise known as 18 U.S.C. Sec. 2339B, creates a broad definition that includes any kind of support for blacklisted groups, encompassing humanitarian aid, medical training, expert advice, other services in just about any form, and of course, political advocacy.

It's this political advocacy portion of the statute that places communication on social media sites, and search engines queries, namely Google, at the top of the list for creating these data-driven networks. Simply looking at a questionable web site, or tweeting something about Syria, Yemen, or anything to do with "radical" subjects, is reason enough for the government to give you a second look. With the powerful software developed by private corporations, it is now much easier to track your day-to-day life, both online and off. While the civil rights and civil liberties issues here are important - due to near-indiscriminate surveillance of ordinary individuals based on generalized algorithm specifications - there's another issue at work here: consolidation of power and control. What we're seeing in the partnership between the feds and companies like Palantir is the nearly complete union of corporation and state. This sets a stage for increasing corruption and consolidation of power into fewer and fewer hands.

It's startling for many reasons, especially when you take into account the attacks on progressive organizations and (non-terrorist) individuals orchestrated by Palantir, and the series of security failures that have occurred on their watch. One such high-profile incident was put into motion by the US Chamber of Commerce, which contracted Palantir "to develop tactics for damaging progressive groups and labor unions, in particular, ThinkProgress, the labor coalition called Change to Win, the SEUI, US Chamber Watch, and StopTheChamber.com."

If spying on unions wasn't bad enough, then failing to put the pieces together leading up to the Boston Marathon bombings on April 15, 2013 would put into question the effectiveness of this type of surveillance and data-mining. It has been widely reported that the Russian Federal Security Service (FSB), successor to the KGB, contacted the CIA in 2011 with information suggesting that Tamerlan Tsarnaev was becoming more radicalized and might have been planning a trip overseas. In response, the CIA submitted a request to the National Counterterrorism Center to add Tsarnaev's name to the Terrorist Identities Datamart Environment (TIDE) list, which functions as the main feeder list to other intelligence agencies. Months after this request was submitted, the FBI conducted an inquiry after it received information from Russian state security with nearly identical information. Palantir software employed by the FBI and CIA and designed to track potential terrorists movement failed to flag the seven-month-long trip Tsarnaev took to Russia. This is the exact scenario Palantir demonstrates to potential clients to exemplify how they structure unstructured data.


Cofounder and CEO of Palantir, Alex Karp explains that Palantir Technologies Inc. is only in the business of developing a "software analytic platform" for the analysis of data, and that they do not provide, "nor do we have any plans to develop," offensive cyber capabilities. If there was ever a clear line between what was open source intelligence and what was considered private or classified information, it no longer exists. The US intelligence community now considers hackers, information security, cyber security and cyber threats top-tier national security threats and have employed private firms to take the fight online. The question is not whether online counter-surveillance techniques will develop to the point where people are once again safe from government scrutiny online, but how long it will take.

Copyright, Truthout.



Read the Rules
[-] 3 points by windyacres (1197) 6 years ago

According to this article, the Russians contacted the CIA about Tamerlan Tsarnev. The CIA added his name to the TIDE list. Later the Russians also notified the FBI and an inquiry was conducted. Then the software used failed to flag the seven month long trip to Russia! How could this be?

I don't understand the conclusion, especially the question posed in the final sentence.

[-] 2 points by LeoYo (5909) 6 years ago

Chris Pyle, Whistleblower on Domestic Spying in '70s, Says Be Wary of Attacks on NSA's Critics

Friday, 14 June 2013 11:15 By Amy Goodman and Juan Gonzalez, Democracy Now! | Video Report


[-] 2 points by LeoYo (5909) 6 years ago

Security Wars: Inside the Military's Big, Messy Fight With Palantir, the Company They Pay to Spy on You Online

Friday, 14 June 2013 00:00 By Stephen Benavides, Truthout | News Analysis


In a House Armed Services Committee hearing at the end of April, California Republican Congressman Duncan Hunter, a former Marine, leveled serious charges against high-level Army officers. He accused them of blocking the use of Palantir technology, the company the military has hired to watch the US public's every online move for signs of potential terrorist activity.

But the House had concerns of its own: In a letter dated August 1, 2012, the House Committee asked why the $2.3 billion had been spent on research and development of the DCGS-A, a global surveillance and intelligence super platform, that despite the mind-boggling sum, failed to work as planned. Reports submitted to House Armed Services Committee outlining serious issues with the global surveillance and intelligence super platform indicated that DCGS-A is "unable to perform simple analytical tasks." More specifically, military intelligence analysts from the Army and Air Force have both expressed that DCGS-A does not "provide intuitive capabilities to see the relationships between a wide variety of disparate data sets of information."

The ongoing fight over the use of Palantir software bubbled over into Congress when the Committee on Oversight and Government Reform sent a letter to then Secretary of Defense Leon Panetta requesting documentation of the forward operations assessment for the Palantir system. Before any technology is deployed by the military, that technology must be vetted in the form of an assessment based on a trial resembling real world situations. Instead of receiving the powerful software system with open arms, Army brass refused to fully implement technology that the FBI and CIA already use to monitor digital communications of US citizens, including surveillance of social media platforms as Facebook and Twitter. Given that the CIA provided the start-up to get Palantir going, there is an interest in having all branches of government implement the same, or similar technology. In the age where terrorists lurk around every corner, and international occupations churn out generation after generation of anti-imperialist youth, consolidating the surveillance and intelligence systems employed would seem to make sense. Now that this private company, Palantir, has become a very successful money-making venture, there appears to be an internal security war going on inside the US government over what system to deploy in international theaters. This is not in the name of the public good, but rather an effort by the US Army to hold its own as other federal agencies like the CIA, FBI, NSA, and DIA increase in power and influence post 9-11.


Palantir Technologies Inc. was a CIA start-up aimed at streamlining the gathering and analysis of massive amounts of data generated from both offline and online human interaction.

Before the FBI and CIA effectively handed over the bulk of their online intelligence gathering and surveillance to software developed by Palantir, there was a focused effort to do something similar in-house nearly 20 years prior. After combat operations ended in Iraq and Kuwait in 1991, the Department of Defense determined that dominance of information technology and intelligence operations would give the US a powerful and strategic advantage over its enemies.

In order to do that, the DoD would have to integrate the nation's Signal Intelligence (SIGINT), Common Imagery (CIGSS), and Imagery Intelligence (IMINT) ground and surface systems into one super system, otherwise known as the Distributed Common Ground System (DCGS AN/GSQ-272). Specifically, DCGS was tasked with the surveillance and reconnaissance of targeted individuals or organizations, as well as the gathering and processing of data, and dissemination of information in an effort to "facilitate Seeing and Knowing on the battlefield."

Since the Internet had yet to be born, intelligence was gathered and disseminated by the predecessors of the U-2 Dragonlady, RQ-4 Global Hawk, MQ-9 Reaper, MQ-1 Predator, and MC-12 Liberty, all different types of manned aircraft "weapons systems," not to mention human intelligence gathered clandestinely in the field. Over the years, DCGS has been deployed in every major foreign conflict, occupation, or intelligence-gathering mission in which the military has been involved.

All of this changed drastically following the attacks on September 11, 2001. Huge amounts of funding poured into creating a slew of new federal agencies. According to a two-year-long investigative piece titled "Top Secret America" completed by the Washington Post, "some 1,271 government organizations and 1,931 private companies work on programs related to counterterrorism, homeland security and intelligence in about 10,000 locations across the United States."

In a July 2010 memo, Deputy Chief of Staff for Intelligence Michael Flynn requested advanced analytical capabilities for US forces stationed in Afghanistan. Flynn stated that "intelligence analysts in theater do not have the tools required to fully analyze the tremendous amounts of information currently available in theater." He went on to say that this intelligence shortfall prohibited commanders from having a full understanding of the operational environment, and that without advanced capabilities, operations would not be as successful as they could be - translating into higher loss of life during combat. In the memo, he goes into specific detail on the type of system needed, and how current tools do not provide "intuitive capabilities to see the relationships between wide varieties of disparate sets of information." This echoes Palantir Technologies' demonstration given to potential clients regarding a fictional terrorist by the name of Mike Fikiri, nearly word for word. Between July 2010 and September 2012, the 3rd Infantry Division stationed in Afghanistan began fusing its current intelligence system with those provided by none other than Palantir Technologies. Some of those new capabilities include using mobile apps and other handheld devices to support combat personnel and commanders in the field by integrating with the Blue Force Tracking (BFT). That's a GPS-enabled system allowing military commanders to know where both friendly and hostile forces are located. More importantly, the software was implemented to locate and destroy roadside bombs and IEDs.

In order to pull this off, servers were provided and installed at no cost by Palantir at Fort Stewart, Georgia, offices, where the 3rd Infantry Division is based. The problem with all of this is that it violated CFR 48 - Federal Acquisitions Regulations, and US Army HQ as a whole was totally unaware and had not signed off on any of it. Because it had been installed under the radar, Army headquarters promptly ordered all of the servers to be shut down and had them removed by the end of September 2012. Army brass even went so far as to have Kim Denver, Army deputy assistant secretary for procurement, issue a cease-and-desist order against Palantir disallowing it from "approaching units and providing goods and services for free." The memo goes on to describe numerous requests from commanders in Afghanistan for a more robust system, and states that there may have been a possible manipulation of Army Test and Evaluation Command (ATEC) assessment reports on the Palantir system. According to the Army's website, ATEC is "the premier test and evaluation organization within DoD and the Army's trusted agent for ensuring that our Warfighters have the right capabilities for success across the entire spectrum of operations." Source documents provided to the House Committee on April 25, 2012, did not match a second version of the report that was created, due to the fact that several survey responses were withheld by Colonel Joseph Martin, commanding officer of the Army Operational Test Command. Colonel Martin apparently gave orders to replace the April 25, 2012, report with the May 25, 2012, version in an obvious attempt to stop the burgeoning tech giant. The memo to Defense Secretary Panetta, issued by the Oversight and Government Reform Committee Chairman, Republican Darrell E. Issa, states "these actions could be construed as limiting positive feedback on use of a more expensive and less effective program.

In August 2011, a military exercise simulating what would happen if North Korea attacked the US resulted in a catastrophic failure of the $2.3 billion DCGS system software, which was developed by Northrop Grumman. During that exercise, the volume of information the system was designed to analyze instead resulted in ten of the 96 hours allotted to the exercise being spent rebooting or outright locked up. If being unable to complete basic functions wasn't enough to infuriate top ranking military officials, a 2.5-minute nomination time frame for bombing targets, expected to take mere seconds, set the stage for private contractors to get a piece of the action.

[-] 4 points by LeoYo (5909) 6 years ago

Palantir seems to be winning the years-long war of attrition, and has secured a cooperative research agreement with the Army, signed in May 2012, as well as having its software purchased for use in Afghanistan shortly thereafter(even after it was at first given away for free, illegally). Taking into account that this little-known tech start-up had an estimated worth of $3 billion as of 2011, not even 6 years after being founded by the CIA, and that it has the influence and know-how to get the majority of the American intelligence community signed on as clients, you have to ask yourself where this is all heading. Having a single private corporation with access to top secret classified information on everything from military troop movements, terrorist watch lists, to everyday movements and communications of common individuals, at what point does it stop? Or is this just the beginning?

Copyright, Truthout.