{"id":74,"date":"2008-09-24T10:47:21","date_gmt":"2008-09-24T15:47:21","guid":{"rendered":"http:\/\/www.webadminblog.com\/?p=74"},"modified":"2008-09-24T10:47:21","modified_gmt":"2008-09-24T15:47:21","slug":"owasp-google-hacking-project-owasp-appsec-nyc-2008","status":"publish","type":"post","link":"https:\/\/www.webadminblog.com\/index.php\/2008\/09\/24\/owasp-google-hacking-project-owasp-appsec-nyc-2008\/","title":{"rendered":"OWASP Google Hacking Project – OWASP AppSec NYC 2008"},"content":{"rendered":"

This presentation is by Christian Heinrich, the project leader for the OWASP “Google Hacking” project.\u00a0 Presentation published on http:\/\/www.slideshare.net\/cmlh\u00a0 Dual licensed under OWASP License and AU Creative Commons 2.5.<\/p>\n

OWASP Testing Guide v3 – Spiders\/Robots\/Crawlers<\/strong><\/span><\/p>\n

1. Automatically traverses hyperlinks<\/p>\n

2. Recursively retrieves content referenced<\/p>\n

Behavior governed by the robots exclusion protocol.\u00a0 New method is <META NAME=”Googlebot” CONTENT=”nofollow”>\u00a0 Not supported by all Robots\/Spiders\/Crawlers.\u00a0 Traditional method is robots.txt located in web root directory.\u00a0 Regular expressions supported by minority only.\u00a0 “User-agent: *” applies to all spiders\/robots\/crawlers or you can specify a specific robot name.\u00a0 Can be intentionally ignored.\u00a0 Not for httpd access control or digital rights management.<\/p>\n

Testing – Robots Exclusion Protocol<\/p>\n

    \n
  1. Sign into Google Webmaster Tools<\/li>\n
  2. On the dashboard, click the URL<\/li>\n
  3. Click “Tools”<\/li>\n
  4. Click “Analyze robots.txt”<\/li>\n<\/ol>\n

    Search Engine Discovery<\/strong><\/span><\/p>\n

    Microsoft Remote Desktop Web Connection: intitle:Remote.Desktop.Web.Connection inurl: tsweb<\/p>\n

    VNC: “VNC Desktop” inurl:5800<\/p>\n

    Outlook Web Access: inurl:”exchange\/logon.asp”<\/p>\n

    Outlook Web Access: intitle:”Microsoft Outlook Web Access – Logon”<\/p>\n

    Adobe Acrobat PDF: filetype:pdf<\/p>\n

    Google caught onto this and is now displaying a “We’re sorry” message with certain searches.\u00a0 To get around, use different search queries that returns overlapping results.<\/p>\n

    Google Advanced Search Operators: “site:” and “cache:”\u00a0 Two ways of using “site:”.\u00a0 EIther as “site:www.google.com” where you get that specific subdomain’s results or “site:google.com” where you get all hostnames and subdomains. Use “cache:www.owasp.org” to display an indexed web page in the google cache.\u00a0 There is also a site operator labeled “Cached” which will do the same thing.<\/p>\n

    You can get updates of the latest relevant Google results (web, news, etc) using Google Alerts.<\/p>\n

    Download Indexed Cache<\/strong><\/span><\/p>\n

    Google SOAP Search API.\u00a0 Query limited to either 10 words or 2048 bytes.\u00a0 One thousand search queries per day and limited to search results within 0-999.\u00a0 Up to 10K possible results from 10 different search queries.<\/p>\n

    $Google_SOA_Search_API -> doGoogleSearch( $key, $q, $start, $maxResults, $filter, $restricts, $safeSearch, $lr, $ie, $oe );<\/p>\n

    See presentation for response.<\/p>\n

    Proof of concept tool is “dic.pl” or “Download Indexed Cache” that downloads the search results.\u00a0 Licensed under the Apache License 2.0.\u00a0 Tool produces a URL and cachedSize response.<\/p>\n

    OWASP Google Hacking Project<\/strong><\/span><\/p>\n

    Tools built using Perl using CPAN Modules SOAP::Lite, Net::Google, and Perl::Critic.\u00a0 Development environmetn is based on Eclipse with EPIC Plug-in.\u00a0 Subversion repository is at code.google.com.<\/p>\n

    Roadmap<\/strong><\/span><\/p>\n

    Upcoming presentations at ToorCon X in San Diego, SecTor 2008 in Toronto, Canada, and RUXCON 2K8 in Sydney, Australia.<\/p>\n

    “TCP Input Text” Proof of Concept<\/p>\n

    “Speak English” Google Translate Workaround<\/p>\n

    Refactor and 3rd Project review of PoC Perl Code with public release at RUXCON 2K8 in November 2008.<\/p>\n

    Check in at code.google.com after RUXCON 2K8<\/p>\n

    4 hr “half day” training course Q1 2009<\/p>\n","protected":false},"excerpt":{"rendered":"

    This presentation is by Christian Heinrich, the project leader for the OWASP “Google Hacking” project.\u00a0 Presentation published on http:\/\/www.slideshare.net\/cmlh\u00a0 Dual licensed under OWASP License and AU Creative Commons 2.5. OWASP Testing Guide v3 – Spiders\/Robots\/Crawlers 1. Automatically traverses hyperlinks 2. Recursively retrieves content referenced Behavior governed by the robots exclusion protocol.\u00a0 New method is <META […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[127],"tags":[76,626,100,132,12,133,102],"class_list":["post-74","post","type-post","status-publish","format-standard","hentry","category-owasp-appsec-nyc-2008","tag-application","tag-conferences","tag-google","tag-hacking","tag-owasp","tag-project","tag-web"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pfI0c-1c","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/posts\/74","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=74"}],"version-history":[{"count":4,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/posts\/74\/revisions"}],"predecessor-version":[{"id":87,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/posts\/74\/revisions\/87"}],"wp:attachment":[{"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=74"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=74"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.webadminblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=74"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}