twill-0.9/0000755000175000001440000000000010735115574010707 5ustar tuserstwill-0.9/doc/0000755000175000001440000000000010735115574011454 5ustar tuserstwill-0.9/doc/advocacy/0000755000175000001440000000000010735115574013245 5ustar tuserstwill-0.9/doc/advocacy/simple-example.txt0000644000175000001440000000055210735115574016732 0ustar tusers#http://www.answermysearches.com/mechanize-useful-tips/229/ twill script: :: add_extra_header User-Agent "Mozilla/5.0 (compatible; MyProgram/0.1)" go http://www.python.org/ show in Python: :: from twill.commands import * add_extra_header('User-Agent', 'Mozilla/5.0 (compatible; MyProgram/0.1)') go("http://python.org/") html = browser.get_html() print html twill-0.9/doc/ANNOUNCE-0.7.1.txt0000644000175000001440000000333210735115574014065 0ustar tusersANNOUNCING twill v0.7.1. twill is a simple Web scripting language built on top of Python and John J. Lee's 'mechanize'. It's designed for automated testing of Web sites, but it should prove useful for anybody who needs to interact with Web sites (especially those using logins and cookies) on the command line or via a script. twill is a reimplementation of Cory Dodt's PBP. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the second public release of twill, version 0.7.1. (Tagline: "Most of the obvious bugs have been fixed.") I'm still looking for general feedback on usability, as well as suggestions on additional use cases. Download directly here: http://darcs.idyll.org/~t/projects/twill-0.7.1.tar.gz Documentation is online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. --- Notable bug fixes and features: * file upload & HTTP basic authentication; * submit w/o submit buttons now works; * better error reporting; * 'quiet', 'interact after scripts', and 'fail-immediately' command-line options. * '-' <== stdin on command line. twill-0.9/doc/ANNOUNCE-0.7.2.txt0000644000175000001440000000312210735115574014063 0ustar tusersANNOUNCING twill v0.7.2. twill is a simple Web scripting language built on top of Python and John J. Lee's 'mechanize'. It's designed for automated testing of Web sites, but it should prove useful for anybody who needs to interact with Web sites (especially those using logins and cookies) on the command line or via a script. twill is a reimplementation of Cory Dodt's PBP. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the third public release of twill, version 0.7.2. (Tagline: "no obvious bugs") I'm still looking for general feedback on usability, as well as suggestions on additional use cases. Download directly here: http://darcs.idyll.org/~t/projects/twill-0.7.2.tar.gz Documentation is online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. --- Notable bug fixes and features: * save_html and HTTP debugging commands; * fixed extend_with bugs; * updated pyparsing to 1.3.2; * first PyPi release; twill-0.9/doc/ANNOUNCE-0.7.3.txt0000644000175000001440000000374510735115574014077 0ustar tusersANNOUNCING twill v0.7.3. twill is a simple Web scripting language built on top of Python and John J. Lee's 'mechanize'. It's designed for automated testing of Web sites, but it should prove useful for anybody who needs to interact with Web sites (especially those using logins and cookies) on the command line or via a script. twill can also now be used for stress-testing and benchmarking of complex sites via the twill-fork script. twill is a reimplementation of Cory Dodt's PBP. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the fourth public release of twill, version 0.7.3. (Tagline: "miscellaneous updates & twill-fork initial implementation") Download directly here: http://darcs.idyll.org/~t/projects/twill-0.7.3.tar.gz Documentation is online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. --- Notable bug fixes and features: * added 'twill-fork' to allow multiprocess execution of twill-scripts for stress-testing & benchmarking; * image 'submit' buttons now allowed (Robert Leftwich); * added 'Accept: text/html' above */* (Nic Ferrier); * added 'run' to execute Python commands (Ed Rahn); * added 'runfile' to execute other scripts (Ed Rahn); * added $variable substitution (Ed Rahn); * added 'setglobal'/'setlocal' to go along with $variable subs; * increased code coverage of tests; twill-0.9/doc/ANNOUNCE-0.7.4.txt0000644000175000001440000000361110735115574014070 0ustar tusersANNOUNCING twill v0.7.4. twill is a simple Web scripting language built on top of Python and John J. Lee's 'mechanize'. It's designed for automated testing of Web sites, but it should prove useful for anybody who needs to interact with Web sites (especially those using logins and cookies) on the command line or via a script. twill can also now be used for stress-testing and benchmarking of complex sites via the twill-fork script. twill is a reimplementation of Cory Dodt's PBP. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the fifth public release of twill, version 0.7.4. (Tagline: "many bugs fixed, nose-based unit tests now work.") Download directly here: http://darcs.idyll.org/~t/projects/twill-0.7.4.tar.gz Documentation is online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. --- Notable bug fixes and features: * better error handling & display; * many, many browsing bugs fixed; * new 'url', 'exit', 'showlinks', 'title', 'config' and 'agent' commands; * 'nose' unit tests and unit-test support infrastructure; c.f. http://www.idyll.org/~t/www-tools/twill.html#unit-testing Thanks go to Tommi Virtanen, James Cameron, sureshvv, William Volkman, and Mike Rovner for patches and bug reports. twill-0.9/doc/ANNOUNCE-0.7.txt0000644000175000001440000000264510735115574013734 0ustar tusersANNOUNCING twill v0.7. twill is a simple Web scripting language built on top of Python and mechanize. It's designed for automated testing of Web sites, but it may be useful for anybody who needs to deal with Web sites (with e.g. logins and cookies) in a non-interactive manner. twill is a reimplementation of Cory Dodt's PBP. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the first public release of twill, version 0.7. (Tagline: "It seems usable to me, but then I'm its author.") With this release, I'm looking for general feedback on usability, as well as suggestions on additional use cases. Download directly here: http://darcs.idyll.org/~t/projects/twill-0.7.tar.gz Documentation is online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. twill-0.9/doc/ANNOUNCE-0.8.1.txt0000644000175000001440000000275510735115574014076 0ustar tusersANNOUNCING twill v0.8.1. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) This is the seventh public release of twill, version 0.8.1. You can install twill with easy_install, or download this release at http://darcs.idyll.org/~t/projects/twill-0.8.1.tar.gz Documentation is included in the .tar.gz and is also online at http://www.idyll.org/~t/www-tools/twill/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- New features: * match_parse extension (sureshvv); * formfile fix for uploading binary files (Tom Parker); * several patches for checkbox/hidden problems (tv); * Referer: headers now correctly added on form submissions (Tom Parker); * documentation reorganization. twill-0.9/doc/ANNOUNCE-0.8.2.txt0000644000175000001440000000344010735115574014067 0ustar tusersANNOUNCING twill v0.8.2. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) This is the eighth public release of twill, version 0.8.2. You can install twill with easy_install, or download this release at http://darcs.idyll.org/~t/projects/twill-0.8.2.tar.gz Documentation is included in the .tar.gz and is also online at http://www.idyll.org/~t/www-tools/twill/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- New features: * added robust-ified form parsing code; * updated to latest mechanize, ClientForm, ClientCookie code; * updated to latest pyparsing code (1.4.1); * updated to latest wsgi_intercept code; * 'set_output' can now be used to redirect twill output specifically; * added execute_script function; * added return values to a number of the simple command-line functions; * fixed bugs in 'showlinks', 'go'; Backwards incompatibility issues: * changed tidy config option names; * wsgi_intercept code now wants a function that does its own app object caching. twill-0.9/doc/ANNOUNCE-0.8.3.txt0000644000175000001440000000253410735115574014073 0ustar tusersANNOUNCING twill v0.8.3. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) This is the ninth public release of twill, version 0.8.3. You can install twill with easy_install, or download this release at http://darcs.idyll.org/~t/projects/twill-0.8.3.tar.gz Documentation is included in the .tar.gz and is also online at http://www.idyll.org/~t/www-tools/twill/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- New features: * fixed minor bug in links parsing; * included BeautifulSoup. (This is basically a quick bug fix release for twill 0.8.2...) twill-0.9/doc/ANNOUNCE-0.8.4.txt0000644000175000001440000000322710735115574014074 0ustar tusersANNOUNCING twill v0.8.4. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) This is the ninth public release of twill, version 0.8.4. You can install twill with easy_install, or download this release at http://darcs.idyll.org/~t/projects/twill-0.8.4.tar.gz Documentation is included in the .tar.gz and is also online at http://www.idyll.org/~t/www-tools/twill/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- Significant changes with 0.8.4: * improved installation docs and simplified installation; * added link checking, requirements processing, and arguments parsing extension modules. * improved extension module handling; added extension module documentation. * fixed memory leaks. * fixed tidy problem on Windows. * line number of scripts printed on traceback. * fixed file:// URLs. * added option for debugging HTTP-EQUIV REFRESH code. twill-0.9/doc/ANNOUNCE-0.8.5.txt0000644000175000001440000000272410735115574014076 0ustar tusersANNOUNCING twill v0.8.5. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) This is the tenth public release of twill, version 0.8.5. You can install twill with easy_install, or download this release at http://darcs.idyll.org/~t/projects/twill-0.8.5.tar.gz Documentation is included in the .tar.gz and is also online at http://twill.idyll.org/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- Significant changes with 0.8.5: * Added 'formfill', 'dirstack', 'csv_iterate', 'dns_check' extension modules; * Fixed several https problems; And, of course, there were the usual major and minor bug fixes. This is a much more robust release than any of the previous ones! twill-0.9/doc/ANNOUNCE-0.8.txt0000644000175000001440000000367010735115574013734 0ustar tusersANNOUNCING twill v0.8. twill is a simple language for testing Web applications. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. twill has an interactive shell, 'twill-sh', and can also run scripts. twill is a reimplementation of Cory Dodt's PBP. It is built on top of Python and John J. Lee's mechanize. A twill script looks like this: # go to the /. login page go http://slashdot.org/login.pl # fill in the form fv 1 unickname test fv 1 upasswd test submit # ok, there's no such account ;). show error HTML. show --- This is the sixth public release of twill, version 0.8. (Tagline: "85% unit tested.") You can install twill with easy_install via PyPi, or download the latest .tar.gz at: http://darcs.idyll.org/~t/projects/twill-0.8.tar.gz Documentation is included in the .tar.gz and is also online at http://www.idyll.org/~t/www-tools/twill.html --- Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill scripts can be recorded with maxq, although scripts may require some hand tweaking at the moment. See the twill documentation for more information. twill does not understand JavaScript, I'm sorry to say. --- New features: * test WSGI Python applications in-process; * Updated to latest versions of mechanize, ClientCookie, ClientForm, and pullparser; * Significant increase in number of unit tests & their code coverage; * Automatic 'tidy' preprocessing when available; * easy_install/eggs now supported; * http-equiv redirect now works; * new commands: - 'formaction' changes form URLs; - 'tidy_ok' checks for 'tidy' correctness; - 'showhistory' command; Special thanks to sureshvv and Simon Buenzli... twill-0.9/doc/ANNOUNCE-0.9b1.txt0000644000175000001440000000423410735115574014155 0ustar tusersANNOUNCING twill v0.9b1. twill is a simple language for browsing the Web. It's designed for automated testing of Web sites, but it can be used to interact with Web sites in a variety of ways. In particular, twill supports form submission, cookies, redirects, and HTTP authentication. This release of twill is officially "Pretty Dang Stable". A twill script to use the "I'm feeling lucky" button on Google looks like this: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show (Note that using this script abuses Google's Terms of Service. So don't.) You can install the latest release of twill with easy_install, easy_install twill or download this release at http://darcs.idyll.org/~t/projects/twill-0.9b1.tar.gz Documentation is included in the .tar.gz and is also online at http://twill.idyll.org/ Miscellaneous details: twill is implemented in Python and uses pyparsing and mechanize. In addition to the existing simple command language, twill can easily be extended with Python. twill also provides a fairly simple and well-documented wrapper around mechanize. twill does not understand JavaScript, I'm sorry to say. --- Significant changes with 0.9b1 * Switched to MIT license. * twill tests now work under Windows and cygwin (switch to subprocess). Fixed a variety of Windows-specific bugs in the process ;). * Complete integration with latest mechanize, including global form access. * Added Seo Sanghyeon's patch to enable form element TAB completion on 'formvalue' command. * twill-sh now gathers scripts from subdirectories. * Added paste.lint test into wsgi_intercept test code. * Cleared up confusing test output when test server didn't run. * Started using the BeautifulSoup code included with mechanize. * Added TWILL_TEST_PORT for running tests on something other than port 8080. * Added Andy Chamber's patch to the formfill extension module. * Fixed DNS tests to be reliant on name records I control. * Removed ClientCookie dependencies. * Defended my PhD. * Added newline '\n' processing in arguments. twill-0.9/doc/ChangeLog0000644000175000001440000006174610735115574013244 0ustar tusers2007-12-27 Titus Brown * Added better error message to 'url' (Matt Wilson). 2007-12-23 Titus Brown * Updated version to 0.9. * Updated license information => MIT in all files, along with Web link and copyright (Will Maier) * Updated tests to work with nose > 0.9.x. 2007-12-04 Titus Brown * Fixed iterator double call in wsgi_intercept (Antti Kaihola). 2007-07-02 C. Titus Brown * Fixed small documentation error in commands (Bill Mill). * Fixed another problem match_parse: made popmatch work properly (Sheryl). 2007-07-01 C. Titus Brown * Moved 'mechanize' to '_mechanize_dist', 'ClientForm' to '_mechanize_dist.ClientForm', as requested for Debian dists. * Fixed a problem with __matchlist__ in match_parse extension (Sheryl). * All raised exceptions now inherit from TwillException (Kevin Turner). * Eliminated FakeResponse and switched to using mechanize.make_response (Jacob Hallan, JJL). * Updated mechanize to latest; includes my two patches to mechanize. 2007-06-04 Titus Brown * Updated mechanize to latest. * Patched subprocess pipe freeing problem (Chris Abraham). (This is a moderately serious bug for long-running programs that use subprocess to try to run programs that don't exist.) My patch modifies run_tidy to remember if subprocess failed because tidy didn't exist. 2007-05-28 Titus Brown * Added 'info' command, docs, and tests. * Removed gopherlib references in mechanize; added test. 2007-03-19 Titus Brown * Fixed bug in my patch to mechanize 'select_form'. * Removed debug 'print' in wsgi_intercept.py. 2007-03-06 Titus Brown * RELEASE 0.9b1. * Changed version to 0.9b1. * Fixed global_form selection on form set (Matt Singer). 2007-02-18 Titus Brown * Added paste.lint tests into wsgi_intercept test code; fixed only problem noted. 2007-02-15 Titus Brown * Fixed browser.get_form to only allow valid form numbers (Mariusz). * Fixed showforms to properly display form numbers (Mariusz). 2007-02-14 Titus Brown * Added a basic check for Web server to tests: tests no longer run if the test Web server isn't available. * Added get_browser() to twill.commands exports. 2007-02-08 Titus Brown * Global forms no longer show up IF EMPTY. 2007-02-04 Titus Brown * Switched to MIT license. 2007-01-29 Titus Brown * Removed BeautifulSoup.py in favor of the one included with mechanize. * Added pages 'show_environ' and 'show_post' to twilltestserver. * Added commands 'add_extra_header', 'show_extra_headers', and 'clear_extra_headers', + tests & docs. (Shaun Walbridge) * Always display proper form number in showforms. 2007-01-24 Titus Brown * Made twill tests work under Windows-cygwin too! 2007-01-23 Titus Brown * Made twill tests work under Windows+cygwin by switching to subprocess. 2007-01-16 Titus Brown * Fixed twill tests (!?) 2007-01-14 Titus Brown * global_form is now form[0] if present. * Deprecated allow_parse_errors config option; isn't needed for latest mechanize. * Updated mechanize and ClientForm to latest svn; patched to pass twill tests. Preserved originals in mechanize/_html.py.jjl, ClientForm.py.jjl. 2007-01-09 Titus Brown * Added complete_fv to formvalue completion patch. * Added formvalue completion patch (Seo Sanghyeon). 2006-11-29 Titus Brown * Fixed test-form.twill to match new output format. * Patched __init__.py to remove UNIX-specific '/' from the sys.path insert statement. (Ramon Felciano) * Patched wsgi_intercept: REMOTE_ADDRESS ==> REMOTE_ADDR. 2006-11-01 Titus Brown * formvalue now returns an informative error message when a file upload field is touched (Matthew Singer) * Added Andy Chamber's formfill patch. * Added tests for Andy Chambers's formfill patch. * Fixed bug in mechanize/_html.py links parsing. 2006-10-26 Titus Brown * Added TWILL_TEST_PORT in for running tests on something other than port 8080. 2006-10-19 Titus Brown * Updated mechanize & ClientForm to svn latest. 2006-10-17 Titus Brown * Fix to BS parsing problem (?) 2006-10-16 Titus Brown * Changed assertions in test-broken-html.py to refer to tidy. * Put a pkg_resources.require('dnspython>=1.4') in for test-dns.py 2006-10-15 Titus Brown * Switched over to using pkg_resources.require for quixote version require/check. * Fixed DNS tests with fake idyll.org hosts (Alex Martelli) 2006-09-20 Titus Brown * Bumped version to 0.9. * Switched references to urllib2 over to mechanize, per JJL. 2006-07-12 Titus Brown * removed ClientCookie dependencies 2006-07-09 Titus Brown * Updated gather code to properly deal with lexical ordering in directory hierarchies. * Finished fixing up the HTML processing code to the latest mechanize. 2006-07-07 Titus Brown * added newline '\n' processing in arguments. still need to test. * added utils.gather_files to gather script files from subdirectories. 2006-06-19 Titus Brown * Fixed name server change in test-dns.twill. * Fixed __match__ info in docs. 2006-06-04 Titus Brown * RELEASE 0.8.5. 2006-05-19 Titus Brown * Added 'formfill' extension module, tests, & docs. (Ben Bangert) * 'debug' now takes boolean (+/-) arguments, too. 2006-05-17 Titus Brown * Bumped version number. * Explicitly str-ified form/field names. (Sebastien Pierre) * Added path mangling issue to 'known problems'. * Modified 'extend_with' import command to also print out location of imported module. * Fixed unnecessary path mangling that introduced the package directory into sys.path; associated check_links idiocy. (Herve Cauwelier) * Fixed test breakage due to amazon.com DNS changes. 2006-05-15 Titus Brown * Bumped version number, fixed import order issue. 2006-05-11 Titus Brown * Added 'check_links' collecting behavior, 'report_bad_links' function. * Added debug logging to csv_iterate. 2006-05-08 Titus Brown * Added dirstack extension module 2006-05-07 Titus Brown * Fixed general https problem ('back' didn't work because the response fp was 'close'd but not wrapped in addinfourl for HTTPS URLs). (re ARINC) * Fixed check_links problem with 'https:' URLs, refactored a tad. (re ARINC) * Fixed wsgi_intercept generator problem, added test. * Fixed a reference to an absolute URL in the tests. 2006-04-29 Titus Brown * Added (re-added!?) test-wsgi-intercept.py test. * 'execute_string' no longer resets context, unlike 'execute_file'. * Fixed a few miscellaneous problems with argparse.get_args; added optional 'require' argument to get_args. * added 'interactive' flag to twill.shell; turned off verbose output for 'extend_with' command if not interactive. 2006-04-24 Titus Brown * Added csv_iterate module. * Fixed argparse to use $arg1...$argN instead of $arg0. 2006-04-23 Titus Brown * Fixed argparse to listen to arguments after '--' only. * Updated 'problems.txt' again with a Windows-specific 'tidy' problem. (aledain) * Refactored '_all_the_same...' code by moving it into utils.py. * browser.get_form() now checks for exact match to form ID. (Uy Do) * browser.get_form_field() now checks for exact match to control ID (Uy Do) * Updated ClientCookie to latest svn. * Added another weird form parsing error:
problems in forms. (David Hancock) * Verified issue with ParseErrors caused by INPUTs in the wrong place. Put in test-broken-html. Cannot fix easily... (Tomi Hautakoski) * Fixed wsgi_intercept/write_fn issue. 2006-04-13 Titus Brown * Changed 'save_html' to default to deriving the filename from the URL (Tristan De Buysscher). 2006-03-30 Titus Brown * Added redirect_error, reset_error to commands.py, and added function 'twill.set_errout'. * Added '-n', '--never-fail' option to 'twill-sh'. * Fixed a problem with ClientForm.AmbiguityError being raised in bad places. * Updated ClientForm to latest. 2006-03-26 Titus Brown * Updated dns_check with Bob Halley's suggestions. * Added docs & (fragile) tests to the dns_check extension module. * Added dns_check.py extension module. No docs/tests yet. 2006-03-25 Titus Brown * Allowed failure to import setuptools in setup.py. 2006-03-23 Titus Brown * Major twill/shell.py patch to allow for dynamic addition of methods to shell objects. * RELEASE 0.8.4. * Improved extension module docstrings. * Added better printout when 'extend_with' is used. * Added argparse extension module (Michele Simionato). 2006-03-22 Titus Brown * Changed __match__ to listen to subgroups. (Leonardo Santagada). * Added Stephen Thorne's fix for multiple checkboxes with the same name/different values. 2006-03-21 Titus Brown * Added 'acknowledge_equiv_refresh' option & added helpful error message on infinite refresh detection (Leonardo Santagada). * Added in Stephen Thorne's fix for readonly-controls printout. * Added in Stephen Thorne's checkbox test & added 'on'/'off' for checkbox stuff. * Added in Stephen Thorne's "with_default_realm" auth patch. 2006-03-17 Titus Brown * Added in Jeff Martin's ${variable} expansion patch. * List values can now be selected/deselected with +/-. * Checkboxes can now be set with True or a non-zero number, and cleared with a False or '0'. * Switched over to using 'require success' instead of code 200 on some scripts. * Fixed a number of unknown options ;). * 'config' now fails on unknown options. 2006-03-15 Titus Brown * Moved utils.journey() function into TwillBrowser class as _journey, refactored, simplified & eliminated _new_page() method. * Added check_links extension module. * Added simple extension module documentation. * Added installation instructions. 2006-03-14 Titus Brown * Added list of known problems. * Added Web testing programs for Perl & PHP to docs. 2006-03-09 Titus Brown * Switched mechanize default encodings over to utf-8 (Terry Peppers, many others!) 2006-03-06 Titus Brown * fixed twill.shell.main() so that -i flag makes interact run even on failure exit. (Kieran Holland) * Added setuptools entry_points magic for twill-sh script. 2006-03-05 Titus Brown * SystemExit problems in the twill shell, fixed. * Added 'redirect_output' and 'reset_output'. (Norman Khine) * Fixed memory leak in reset_browser(). (Alexander Shvedunov) * Increased test coverage overall to 85%. * Now test the shell code fairly well. * Cleaned up some of the stdout issues with nose. * fixed buggishness in shell code. * Made sure to close fp in tidy code. 2006-02-28 Titus Brown * runfile now allows multiple filenames properly. 2006-02-21 Titus Brown * TwillAssertionError no longer subclasses AssertionError. * execute_command now takes a cmdinfo argument so that line number of input can be traced in errors. 2006-02-16 Titus Brown * Fixed 'run_tidy' >& error on windows. * Fixed "file:" URLs for which result.code doesn't exist. 2006-02-09 Titus Brown * Added 'equiv-refresh' debug option. 2006-02-08 Titus Brown * 0.8.3 RELEASE * fixed minor links parsing bug (seek(0) prior to parsing!) * renamed wwwsearch/ to other_packages/. * added BeautifulSoup. 2006-02-06 Titus Brown * 0.8.2 RELEASE. * Documented config options. * Added in parse_errors stuff: config parse options now all work. * Wedged in all of the mechanize parse Factory refactoring. * Refactored a bit to use Factory() amalgamation. In particular, removed title() override in PatchedMechanizeBrowser; built a Factory subclass in utils; and removed external references to TidyAware code. 2006-02-05 Titus Brown * Added 'set_output' command to redirect all output elsewhere. * Updated pyparsing to 1.4.1. 2006-02-04 Titus Brown * added execute_string. * Updated wsgi_intercept to latest (non-caching) version. * Removed test-wsgi-intercept code. 2006-02-01 Titus Brown * Added ignore_errors functioning into wwwsearch code; now, when ignore_errors=True on form parsing, parse errors are ignored. * Renamed 'do_run_tidy' to 'use_tidy', and 'tidy_should_exist' to 'require_tidy'. * Fixed redirect error once and for all by switching to using the *correct* Request class. 2006-01-23 Titus Brown * Added extensions/ __init__.py * Removed special setup cmdclass. 2006-01-22 Titus Brown * Removed space mangling in favor of JJL's fix in mechanize. * Fixed multiple 'showlinks' problem (in _mechanize.py). * Catch all parsing exceptions. (Kieran Holland) * Added return values to a number of functions in commands.py, documented appropriately. * Added '?' behavior for 'go', so that 'go ?name=value' works. (Sebastjan Trepca) * Updated pyparsing to 1.4p1 version. * Updated mechanize/ClientCookie/ClientForm. * Removed ez_setup dependency. 2005-12-19 Titus Brown * Added tv@debian.org's 'find' regexp flags patch. * Added tv@debian.org's quoted patch. * Added tv@debian.org's patch to widen showforms field names. * Added tv@debian.org's patch to fix checkbox/submit multiplicity errors. * Added Jason Pellerin's tests patch. 2005-12-11 Titus Brown * Added twill.get_browser(); deprecated twill.get_browser_state(). * Broke mechanize stuff out into _browser.py. * Removed browser.url() in favor of get_url(). 2005-12-10 Titus Brown * RELEASE 0.8.1. * Added Tom Parker's patch for multiple identical submit buttons. 2005-12-08 Titus Brown * fixed wsgi_intercept start_response params: added exc_info. * added 'sleep' argument to TestInfo init. 2005-12-06 Titus Brown * Added sureshvv's commands in as 'match_parse' extension. * Moved extensions/ into twill/extensions; modified sys.path accordingly in package import. 2005-12-05 Titus Brown * Fixed a problem where tidy was occasionally mangling long titles with unnecessary newlines. * Fixed a bug in PatchedMechanizeBrowser.title() where seek(0) was not getting called. 2005-12-03 Titus Brown * formfile now uses 'rb' to open files. (Tom Parker) * local namespace now automatically created if it doesn't exist. * Applied tv's patches for checkbox/hidden problems and incorrect error messages. * Fixed form submission to add referer headers; updated tests. (Tom Parker) * Fixed tests: setup -> setup_module, teardown -> teardown_module. 2005-11-29 Titus Brown * RELEASE 0.8. * Additional coverage and some miscellaneous fixes. 2005-11-28 Titus Brown * Made http-equiv=refresh stuff work by overriding it in utils.py. My fix is probably bad. oh well, shouldn't make a difference... 2005-11-27 Titus Brown * Verified that eggs stuff now "just works". * Put in a fix for an HTTP Basic Auth bug (that is fixed in Py 2.5!) * Added in a test of 'formaction'. * Added in a bunch of form tests to test-form, fixed bugs discovered thereby. * Added in sureshvv's formaction/fa patch. * Added in docs re 'tidy'. * Put in a rough 'tidy' implementation by wrapping FormsFactory and LinksFactory.parser_class. * Removed 'get_title' from RequestWrapper, moved to using MechanizeBrowser.title(). * Added 'utils.run_tidy' function & 'test-tidy.py' under tests/. * Moved the journey-exception-wrapping into the mechanize mech_open function; this fixes a bug reported by sureshvv. Augmented test-back.twill to test this, and added a 'plaintext' function to the twill test server. 2005-11-20 Titus Brown * Switched to using pullparser to extract title in utils. * Increased code coverage of unit tests significantly. 2005-11-19 Titus Brown * Fixed 'code' command to display non-numeric codes from get_code() (which may return None). 'code' itself still requires an integer. * Fixed test-wsgi-intercept.py to init global/local dictionaries. * Added 'test-find.*' * Set __match__ to matching string in find, url, and title. 2005-11-18 Titus Brown * Version upgraded to 0.8. * Removed ez_setup hooks for the moment, because they interfered with build/install. * Finished updating to latest mechanize code. whew. * Miscellaneous fixes to browser.py. * Miscellaneous fixes to ClientCookie/_Util.py. * Added 'showhistory' command. * WSGI intercept stuff is now much cleaner. Could be even cleaner, yes... * Updated ClientForm/ClientCookie/pullparser. 2005-11-17 Titus Brown * substantially updated README with links to other tools. * KeyboardInterrupts don't error out in twill-sh; they just force a silent exit. * Added a code 500 test to 'test-http-codes'. 2005-11-15 Titus Brown * Fixed failure to seek(0) before parsing response; problem in _parse_html. It broke 'back()' behavior for forms. Added test-back.{py,twill}. * Test cleanup: add try: finally: around __main__ stuff, and add 'initial_url' args. * Added 'advanced docs' section. * Added http_proxy documentation. * Fixed problems caused by failure to construct a valid result object when HTTP errors (e.g. 404) happen. Tested by test-http-codes.py. * Added test-http-codes, cleaned up testlib a bit. * Fixed a stupid bug in parse.py with 'local_dict' != 'locals_dict'. 2005-11-14 Titus Brown * Added small section to README about 'nose' unit tests. * Cleaned up the 'myhttplib' WSGI intercept code & added a test. 2005-11-11 Titus Brown * Downgraded 0.8 version to 0.7.4. RELEASE. 2005-11-10 Titus Brown * Moved 'test/' to 'tests/', added 'test-multisub.py', moved 'twillserver.py' to 'twilltestserver.py', and fixed it to run simple_server when executed from the command line. * Fixed two form submission bugs, pointed out by Mike Rovner. 2005-11-09 Titus Brown * Added William Volkman's patch fixing 'agent'. 2005-11-06 Titus Brown * Updated 'formvalue' and 'submit' help docs. 2005-11-05 Titus Brown * Added 'config' command, 'readonly_controls_writeable' option. (James Cameron bug/fix) * Updated version numbers to 0.8. * Added command 'showlinks'. * Changed 'state' variable to 'browser' in commands.py 2005-11-04 Titus Brown * Put test/ tests into a nosetest-compatible format. Yay, unit tests! 2005-10-30 Titus Brown * Now catch errors from __ and $ eval failure (i.e. not in any namespace) and reset the evaluation to just be the original name. (bug reported by sureshvv). 2005-10-29 Titus Brown * Cleared content-type header in wwwsearch.zip, urllib2_support.py, class HTTPRedirectHandler, redirect_request. This fixes a bug I noticed when testing collar: redirects inappropriately retain content-type, and this breaks Quixote, at the least, when enctype multipart/form-data is used in a POST prior to the redirect. (CTB: this should be dealt with more generally.) * Fixed back() behavior in _mechanize. (I probably broke it in the first place, sigh.) Now accords with latest trunk _mechanize.py. * Fixed order of __url__ setting in parse.execute_command. * Added 'url' and 'exit' commands. * Added 'debug twill' to set printout of twill commands as they are executed. * Attempted to add http_auth testing, but failed. (non-functional code left in test/) * Overrode 'view_html' bool function in PatchedMechanizeBrowser. It now assumes that no content-type means text/html, and also allows 'text/xml' to be used. (James Cameron's issue.) * moved _TwillBrowserState into browser.py, named it TwillBrowser(). * twill-sh now prints out the exception str when executing files, even when 'fail' flag is not set. 2005-10-18 Titus Brown * fixed bug in get_form_field that inverted the priorities of form/field searches. * added documentation for add_auth, form/field identification. 2005-10-09 Titus Brown * Added tv@debian.org's patch: add 'title' assertion, and show error messages when TwillAssertions are caught. 2005-10-07 Titus Brown * Updated to version 0.7.3 in the code; RELEASE. * Added 'twill-fork' script for stress-testing. 2005-10-01 Titus Brown * Increased code coverage of tests by adding in a few commands to test_twill.twill. * Added Robert Leftwich's patch to allow the use of ImageControl for 'submit'. 2005-09-20 Titus Brown * Added Nic Ferrier's patch to request text/html first. 2005-09-18 Titus Brown * Switched to using eval for $variable substitution. * Created namespaces.py, moved local/global dictionary stuff in there. Created local namespace stack; namespaces now mean something ;). * Added setglobal and setlocal, then agonized about local/global dictionaries. * Added Ed Rahn's patch: run, runfile, and $variable substition. Documented appropriately. 2005-09-10 Titus Brown * Updated version to 0.7.2. RELEASE. * Updated setup.py to contain full Pypi info & Trove classification. * Made sure current directory was in sys.path on load; useful for extend_with! * Added debug http functionality. * Added save_html . 2005-09-03 Titus Brown * Fixed extend_with to update the interactive shell. 2005-08-20 Titus Brown * Updated pyparsing.py to 1.3.2. 2005-06-08 Titus Brown * Added 'add_auth' command for HTTP basic authentication. * Added 'formfile' command for file uploads. * Added 'test/README.txt' for testing docs. * Applied Joeri van Ruth's patch to make '-' as a filename read from stdin. 2005-06-06 Titus Brown * Now checks for exact matches to fieldnames before trying out regexps. * Submit w/o submit buttons now working. * Fixed quoted handling of __stuff__. * More intelligent pyparsing stuff from Paul McGuire; in particular, comments on the ends of lines are allowed, and '#' in arguments is disallowed. 2005-06-04 Titus Brown * added a couple of command line options using 'optparse'. * added __url__ ==> current URL. * added some of Paul McGuire's suggestions for cleaning up the pyparsing code. * better error reporting on all exceptions while executing file. * Fixed a relative URL issue: twill now tries several URLs in order, only prepending 'http://' when it's tried other URLs first. Added test. * Added Joeri van Ruth's patches: new sleep command, and transmit value of submit buttons on form submission. Added tests for the latter. Documented. 2005-05-26 Titus Brown * all files specified on command-line now executed. * emacs backup files (filename~) now skipped by twill-sh. * error output on script execution now includes line number & filename. 2005-05-25 Titus Brown * added reset_state; run it on each execute_file. 2005-05-24 Titus Brown * 'go domainname' now works as expected. * Made various form-related submit() errors clearer. * Fixed problem with spaces in link *text*. (Actually a more general bug that caused link searches to fail in most cases.) 2005-05-23 Titus Brown * Fixed problem with spaces in links/URLs. twill-0.9/doc/LICENSE.txt0000644000175000001440000000220010735115574013271 0ustar tusersThe MIT License, http://www.opensource.org/licenses/mit-license.php twill is Copyright (c) 2005, 2006, 2007 by C. Titus Brown. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. twill-0.9/doc/Makefile0000644000175000001440000000031510735115574013113 0ustar tusersall: index.html commands.html developer.html python-api.html testing.html \ other.html browsing.html examples.html problems.html extensions.html \ install.html %.html : %.txt rst2html.py $< $@ twill-0.9/doc/browsing.html0000644000175000001440000001435710735115574014206 0ustar tusers Web browsing with twill

Web browsing with twill

twill strives to be a complete implementation of a Web browser, omitting only JavaScript support. It includes support for cookies, basic authentication, and most (all?) HTTP trickery, including HTTP-EQUIV redirects. Please let me know if you find a situation where it doesn't work!

twill implements a variety of commands. With the built-in language, you can do things like go to a specific URL; follow links; fill out forms and submit them; save, load, and delete cookies; and change the agent string. You can also easily extend twill with new and specialized Python commands.

Using twill interactively

twill-sh lets you interactively browse the Web. It features built-in help, e.g. "help go" will describe the command 'go' to you; command-line completion with TAB; and history browsing (UP/DOWN arrows).

Proxy servers

twill understands the http_proxy environment variable generically used to set proxy server information. To use a proxy in UNIX or Windows, just set the http_proxy environment variable, e.g.

% export http_proxy="http://www.someproxy.com:3128"

or

% setenv http_proxy="http://www.someotherproxy.com:3148"

Recording scripts

Writing twill scripts is boring. One simple way to get at least a rough initial script is to use the maxq recorder to generate a twill script. maxq acts as an HTTP proxy and records all HTTP traffic; I have written a simple twill script generator for it. The script generator and installation docs are included in the twill distribution under the directory maxq/.

A more recent option is to use the Firefox extension TestGen4Web, which will record your Web browsing in a standard format. Matt Harrison is working on converter.

Running and using tidy

The tidy program does a nice job of producing correct HTML from mangled, broken, eeevil Web pages. By default, twill will run pages through tidy before processing them. This is on by default because the Python libraries that parse HTML are very bad at dealing with incorrect HTML, and will often return incorrect results on "real world" Web pages.

To disable this feature, set config do_run_tidy 0.

If tidy is not installed, twill will silently ignore it. It may be desirable to require a functioning tidy installation; so, to fail when tidy isn't installed, set config tidy_should_exist 1.

See the tidy page for more information on tidy.

Miscellaneous implementation details

  • twill ignores robots.txt.
  • http-equiv=refresh headers are handled immediately, independent of the 'pause' component of the 'content' attribute.
  • twill does not understand javascript.

Robust parsing of HTML

Often (usually?) you don't control the HTML on Web pages you visit. There are several different configuration options that control how lenient twill is in parsing Web pages.

The three basic options to look at are 'use_tidy', 'use_BeautifulSoup', and 'allow_parse_errors'. By default, twill will attempt to use the 'tidy' HTML preprocessor program to clean up HTML before parsing the page. twill will also attempt to use the BeautifulSoup parser, if installed, to parse the page. And, finally, for really miserable pages, the form parsing code will ignore parse errors as much as possible. To turn on strict HTML parsing, set the config options like so:

config use_tidy 0
config use_BeautifulSoup 0
config allow_parse_errors 0

(By default, all options are set to '1'.)

You can set 'require_tidy' and 'require_BeautifulSoup' to require that tidy and BeautifulSoup be installed for your script.

The 'tidy_ok' command can be used to assert that tidy reports no warnings or errors.

twill-0.9/doc/browsing.txt0000644000175000001440000001032410735115574014047 0ustar tusers======================= Web browsing with twill ======================= twill strives to be a complete implementation of a Web browser, omitting only JavaScript support. It includes support for cookies, basic authentication, and most (all?) HTTP trickery, including HTTP-EQUIV redirects. Please `let me know`_ if you find a situation where it doesn't work! twill implements a variety of commands_. With the built-in language, you can do things like go to a specific URL; follow links; fill out forms and submit them; save, load, and delete cookies; and change the agent string. You can also easily extend twill with new and specialized Python commands. .. _let me know: twill@lists.idyll.org .. _commands: commands.html Using twill interactively ~~~~~~~~~~~~~~~~~~~~~~~~~ twill-sh lets you interactively browse the Web. It features built-in help, e.g. "help go" will describe the command 'go' to you; command-line completion with TAB; and history browsing (UP/DOWN arrows). Proxy servers ~~~~~~~~~~~~~ twill understands the ``http_proxy`` environment variable generically used to set proxy server information. To use a proxy in UNIX or Windows, just set the ``http_proxy`` environment variable, e.g. :: % export http_proxy="http://www.someproxy.com:3128" or :: % setenv http_proxy="http://www.someotherproxy.com:3148" Recording scripts ~~~~~~~~~~~~~~~~~ Writing twill scripts is boring. One simple way to get at least a rough initial script is to use the maxq_ recorder to generate a twill script. maxq_ acts as an HTTP proxy and records all HTTP traffic; I have written a simple twill script generator for it. The script generator and installation docs are included in the twill distribution under the directory ``maxq/``. A more recent option is to use the Firefox extension TestGen4Web_, which will record your Web browsing in a standard format. Matt Harrison is working on converter. .. _converter: http://panela.blog-city.com/generate_twill_scripts_and_mechanize_unittests_from_testgen4.htm .. _TestGen4Web: http://developer.spikesource.com/wiki/index.php/Projects:TestGen4Web Running and using tidy ~~~~~~~~~~~~~~~~~~~~~~ The ``tidy`` program does a nice job of producing correct HTML from mangled, broken, eeevil Web pages. By default, twill will run pages through ``tidy`` before processing them. This is on by default because the Python libraries that parse HTML are very bad at dealing with incorrect HTML, and will often return incorrect results on "real world" Web pages. To disable this feature, set ``config do_run_tidy 0``. If ``tidy`` is not installed, twill will silently ignore it. It may be desirable to *require* a functioning ``tidy`` installation; so, to fail when ``tidy`` *isn't* installed, set ``config tidy_should_exist 1``. See the `tidy page`_ for more information on ``tidy``. Miscellaneous implementation details ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * twill ignores robots.txt. * http-equiv=refresh headers are handled immediately, independent of the 'pause' component of the 'content' attribute. * twill does not understand javascript. .. _maxq: http://maxq.tigris.org/ .. _tidy page: http://tidy.sourceforge.net/ Robust parsing of HTML ~~~~~~~~~~~~~~~~~~~~~~ Often (usually?) you don't control the HTML on Web pages you visit. There are several different configuration options that control how lenient twill is in parsing Web pages. The three basic options to look at are 'use_tidy', 'use_BeautifulSoup', and 'allow_parse_errors'. By default, twill will attempt to use the 'tidy' HTML preprocessor program to clean up HTML before parsing the page. twill will also attempt to use the BeautifulSoup_ parser, if installed, to parse the page. And, finally, for really miserable pages, the form parsing code will ignore parse errors as much as possible. To turn on strict HTML parsing, set the config options like so: :: config use_tidy 0 config use_BeautifulSoup 0 config allow_parse_errors 0 (By default, all options are set to '1'.) You can set 'require_tidy' and 'require_BeautifulSoup' to require that tidy and BeautifulSoup be installed for your script. The 'tidy_ok' command can be used to assert that tidy reports no warnings or errors. .. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/ twill-0.9/doc/commands.html0000644000175000001440000004373610735115574014160 0ustar tusers twill language reference

twill language reference

The following commands are built into twill. Note that all text after a '#' is ignored as a comment, unless it's in a quoted string.

Browsing

go <url> -- visit the given URL. The Python function returns the final URL visited, after all redirects.

back -- return to the previous URL. The Python function returns that URL, if any.

reload -- reload the current URL. The Python function returns that URL, if any.

follow <link name> -- follow the given link. The Python function returns the final URL visited, after all redirects.

Assertions

code <code> -- assert that the last page loaded had this HTTP status, e.g. code 200 asserts that the page loaded fine.

find <regexp> -- assert that the page contains this regular expression. The variable __match__ is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned.

notfind <regexp> -- assert that the page does not contain this regular expression.

url <regexp> -- assert that the current URL matches the given regexp. The variable __match__ is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned.

title <regexp> -- assert that the title of this page matches this regular expression. The variable __match__ is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned.

Display

echo <string> -- echo the string to the screen.

info -- display information about the current page.

redirect_output <filename> -- append all twill output to the given file.

reset_output -- display all output to the screen.

save_html [<filename>] -- save the current page's HTML into a file. If no filename is given, derive the filename from the URL.

show -- show the current page's HTML. When called from Python, this function will also return a string containing the HTML.

showlinks -- show all of the links on the current page. When called from Python, this function returns a list of the link objects.

showforms -- show all of the forms on the current page. When called from Python, this function returns a list of the forms.

showhistory -- show the browser history. When called from Python, this function returns the history.

Forms

submit [<n>] -- click the n'th submit button, if given; otherwise submit via the last submission button clicked; if nothing clicked, use the first submit button on the form. See details on form handling for more information.

formvalue <formnum> <fieldname> <value> --- set the given field in the given form to the given value. For read-only form widgets/controls, the click may be recorded for use by submit, but the value is not changed unless the 'config' command has changed the default behavior. See 'config' and details on form handling for more information on the 'formvalue' command.

For list widgets, you can use 'formvalue <formnum> <fieldname> +value' or 'formvalue <formnum> <fieldname> -value' to select or deselect a particular value.

fv -- abbreviation for 'formvalue'.

formaction <formnum> <action> -- change the form action URL to the given URL.

fa -- abbreviation for 'fa'.

formclear -- clear all values in the form.

formfile <formspec> <fieldspec> <filename> [ <content_type> ] -- attach a file to a file upload button by filename.

Cookies

save_cookies <filename> -- save current cookie jar into a file.

load_cookies <filename> -- replace current cookie jar with file contents.

clear_cookies -- clear all of the current cookies.

show_cookies -- show all of the current cookies.

Debugging

debug <what> <level> -- turn on or off debugging/tracing for
various functions. The first argument is either 'http' to show HTTP headers, 'equiv-refresh' to test HTTP EQUIV-REFRESH headers, or 'commands' to show twill commands. The second argument is '0' for off, '1' for on.

Variable handling

setglobal <name> <value> -- set variable <name> to value <value> in global dictionary. The value can be retrieved with '$value'.

setlocal <name> <value> -- set variable <name> to value <value> in local dictionary. The value can be retrieved with '$value'.

The local dictionary is file-specific, while the global module is general to all the commands. Local variables will override global variables if they exist.

Note that you can do variable interpolation in strings with ${var}, e.g.

setglobal a 1
setglobal b 2

fv thisform thatfield "${a}${b}"

Other commands

tidy_ok -- check to see if 'tidy' runs on this page without any errors or warnings. (tidy is very stringent -- you've been warned!)

exit [<code>] -- exit with the given integer code, if specified. 'code' defaults to 0.

run <command> -- execute the given Python command.

runfile <file1> [ <file2> ... ] -- execute the given files.

agent -- set the browser's "User-agent" string.

sleep [<seconds>] -- sleep the given number of seconds. Defaults to 1 second.

reset_browser -- reset the browser.

extend_with <module> -- import commands from Python module. This acts like from <module> import * does in Python, so e.g. a function fn in extmodule would be available as fn. See examples/extend_example.py for an example.

getinput <prompt> -- get keyboard input and store it in __input__. When called from Python, this function returns the input value.

getpassword <prompt> -- get silent keyboard input and store it in __password__. When called from Python, this function returns the input value.

add_auth <realm> <uri> <user> <password> -- add HTTP Basic Authentication information for the given realm/URI combination. For example,

add_auth IdyllStuff http://www.idyll.org/ titus test

tells twill that a request from the authentication realm "IdyllStuff" under http://www.idyll.org/ should be answered with username 'titus', password 'test'. If the 'with_default_realm' option is set to True, ignore 'realm'.

config [<key> [<value>]] -- show/set configuration options.

add_extra_headers <name> <value> -- add an extra HTTP header to each HTTP request.

show_extra_headers -- show the headers being added to each HTTP request.

clear_extra_headers -- clear the headers being added to each HTTP request.

Special variables

__input__ -- result of last getinput.

__match__ -- matched text from last find, title, or url.

__password__ -- result of last getpassword.

__url__ -- current URL.

Details on form handling

Both the formvalue (or fv) and submit commands rely on a certain amount of implicit cleverness to do their work. In odd situations, it can be annoying to determine exactly what form field formvalue is going to pick based on your field name, or what form & field submit is going to "click" on.

Here is the pseudocode for how formvalue and submit figure out what form to use (function twill.commands.browser.get_form):

for each form on page:
    if supplied regexp pattern matches the form name, select

if no form name, try converting to an integer N & using N-1 as
an index into the list or forms on the page (i.e. form 1 is the
first form on the page).

Here is the pseudocode for how formvalue and submit figure out what form field to use (function twill.commands.browser.get_form_field):

search current form for control name with exact match to fieldname;
if single (unique) match, select.

if no match, convert fieldname into a number and use as an index, if
possible.

if no match, search current form for control name with regexp match to fieldname;
if single (unique) match, select.

if *still* no match, look for exact matches to submit-button values.
if single (unique) match, select.

Here is the pseudocode for submit:

if a form was _not_ previously selected by formvalue:
   if there's only one form on the page, select it.
   otherwise, fail.

if a field is not explicitly named:
   if a submit button was "clicked" with formvalue, use it.
   otherwise, use the first submit button on the form, if any.
otherwise:
   find the field using the same rules as formvalue

finally, if a button has been picked, submit using it;
otherwise, submit without using a button
twill-0.9/doc/commands.txt0000644000175000001440000002131610735115574014021 0ustar tusers======================== twill language reference ======================== The following commands are built into twill. Note that all text after a '#' is ignored as a comment, unless it's in a quoted string. Browsing ======== **go** ** -- visit the given URL. The Python function returns the final URL visited, after all redirects. **back** -- return to the previous URL. The Python function returns that URL, if any. **reload** -- reload the current URL. The Python function returns that URL, if any. **follow** ** -- follow the given link. The Python function returns the final URL visited, after all redirects. Assertions ========== **code** ** -- assert that the last page loaded had this HTTP status, e.g. ``code 200`` asserts that the page loaded fine. **find** ** -- assert that the page contains this regular expression. The variable ``__match__`` is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned. **notfind** ** -- assert that the page *does not* contain this regular expression. **url** ** -- assert that the current URL matches the given regexp. The variable ``__match__`` is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned. **title** ** -- assert that the title of this page matches this regular expression. The variable ``__match__`` is set to the first matching subgroup (or the entire matching string, if no subgroups are specified). When called from Python, the matching string is returned. Display ======= **echo** ** -- echo the string to the screen. **info** -- display information about the current page. **redirect_output** ** -- append all twill output to the given file. **reset_output** -- display all output to the screen. **save_html** *[]* -- save the current page's HTML into a file. If no filename is given, derive the filename from the URL. **show** -- show the current page's HTML. When called from Python, this function will also return a string containing the HTML. **showlinks** -- show all of the links on the current page. When called from Python, this function returns a list of the link objects. **showforms** -- show all of the forms on the current page. When called from Python, this function returns a list of the forms. **showhistory** -- show the browser history. When called from Python, this function returns the history. Forms ===== **submit** *[]* -- click the n'th submit button, if given; otherwise submit via the last submission button clicked; if nothing clicked, use the first submit button on the form. See `details on form handling`_ for more information. **formvalue** * * --- set the given field in the given form to the given value. For read-only form widgets/controls, the click may be recorded for use by **submit**, but the value is not changed unless the 'config' command has changed the default behavior. See 'config' and `details on form handling`_ for more information on the 'formvalue' command. For list widgets, you can use 'formvalue +value' or 'formvalue -value' to select or deselect a particular value. **fv** -- abbreviation for 'formvalue'. **formaction** * * -- change the form action URL to the given URL. **fa** -- abbreviation for 'fa'. **formclear** -- clear all values in the form. **formfile** * [ ]* -- attach a file to a file upload button by filename. Cookies ======= **save_cookies** ** -- save current cookie jar into a file. **load_cookies** ** -- replace current cookie jar with file contents. **clear_cookies** -- clear all of the current cookies. **show_cookies** -- show all of the current cookies. Debugging ========= **debug** ** ** -- turn on or off debugging/tracing for various functions. The first argument is either 'http' to show HTTP headers, 'equiv-refresh' to test HTTP EQUIV-REFRESH headers, or 'commands' to show twill commands. The second argument is '0' for off, '1' for on. Variable handling ================= **setglobal** * * -- set variable to value in global dictionary. The value can be retrieved with '$value'. **setlocal** * * -- set variable to value in local dictionary. The value can be retrieved with '$value'. The local dictionary is file-specific, while the global module is general to all the commands. Local variables will override global variables if they exist. Note that you can do variable interpolation in strings with ${var}, e.g. :: setglobal a 1 setglobal b 2 fv thisform thatfield "${a}${b}" Other commands ============== **tidy_ok** -- check to see if 'tidy' runs on this page without any errors or warnings. (`tidy` is very stringent -- you've been warned!) **exit** *[]* -- exit with the given integer code, if specified. 'code' defaults to 0. **run** ** -- execute the given Python command. **runfile** * [ ... ]* -- execute the given files. **agent** -- set the browser's "User-agent" string. **sleep** *[]* -- sleep the given number of seconds. Defaults to 1 second. **reset_browser** -- reset the browser. **extend_with** ** -- import commands from Python module. This acts like ``from import *`` does in Python, so e.g. a function ``fn`` in ``extmodule`` would be available as ``fn``. See *examples/extend_example.py* for an example. **getinput** ** -- get keyboard input and store it in ``__input__``. When called from Python, this function returns the input value. **getpassword** ** -- get *silent* keyboard input and store it in ``__password__``. When called from Python, this function returns the input value. **add_auth** * * -- add HTTP Basic Authentication information for the given realm/URI combination. For example, :: add_auth IdyllStuff http://www.idyll.org/ titus test tells twill that a request from the authentication realm "IdyllStuff" under http://www.idyll.org/ should be answered with username 'titus', password 'test'. If the 'with_default_realm' option is set to True, ignore 'realm'. **config** [** [**]] -- show/set configuration options. **add_extra_headers** ** ** -- add an extra HTTP header to each HTTP request. **show_extra_headers** -- show the headers being added to each HTTP request. **clear_extra_headers** -- clear the headers being added to each HTTP request. Special variables ================= **__input__** -- result of last **getinput**. **__match__** -- matched text from last **find**, **title**, or **url**. **__password__** -- result of last **getpassword**. **__url__** -- current URL. Details on form handling ======================== .. _details on form handling: Both the `formvalue` (or `fv`) and `submit` commands rely on a certain amount of implicit cleverness to do their work. In odd situations, it can be annoying to determine exactly what form field `formvalue` is going to pick based on your field name, or what form & field `submit` is going to "click" on. Here is the pseudocode for how `formvalue` and `submit` figure out what form to use (function `twill.commands.browser.get_form`):: for each form on page: if supplied regexp pattern matches the form name, select if no form name, try converting to an integer N & using N-1 as an index into the list or forms on the page (i.e. form 1 is the first form on the page). Here is the pseudocode for how `formvalue` and `submit` figure out what form field to use (function `twill.commands.browser.get_form_field`):: search current form for control name with exact match to fieldname; if single (unique) match, select. if no match, convert fieldname into a number and use as an index, if possible. if no match, search current form for control name with regexp match to fieldname; if single (unique) match, select. if *still* no match, look for exact matches to submit-button values. if single (unique) match, select. Here is the pseudocode for `submit`:: if a form was _not_ previously selected by formvalue: if there's only one form on the page, select it. otherwise, fail. if a field is not explicitly named: if a submit button was "clicked" with formvalue, use it. otherwise, use the first submit button on the form, if any. otherwise: find the field using the same rules as formvalue finally, if a button has been picked, submit using it; otherwise, submit without using a button twill-0.9/doc/ctb-notes.txt0000644000175000001440000000072410735115574014116 0ustar tusers% darcs tag --checkpoint -m release-0.8 -- tag release-0.8, and allow for --partial get. % darcs dist -d twill-0.8.3 where to change version number: setup.py, version & download URL; twill/__init__.py doc/index.txt, version, download URL, and date; for release: * copy doc/ over to twill/. * tag; * generate .tar.gz * generate egg; * update pypi (setup.py register); * announce on clp, twill@lists.idyll.org, web-sig@python.org; twill-0.9/doc/default.css0000644000175000001440000000114610735115574013614 0ustar tusersbody,p,table,li,h1,h2,h3,hr { font-family: verdana, arial, 'sans serif'; font-size: 95% } strong { font-family: verdana, arial, 'sans serif'; font-size: 85% } h1.title { font-size: 110% } body { #ffffff; } a:link {COLOR: navy;} a:visited {COLOR: firebrick;} a:active {COLOR: seashell;} a:hover {COLOR: navy;} div { margin:0; padding:0 .25em 0 0; /* text-align: right border:1px solid #aaa; background-color:#ececec; */ } pre.literal-block, pre.doctest-block { margin-left: 2em ; margin-right: 2em ; background-color: #ffffff; border: thin black solid; padding: 5px; } twill-0.9/doc/developer.html0000644000175000001440000002560310735115574014335 0ustar tusers Developing twill

Developing twill

twill is entirely written in Python. You will need Python 2.3 or later to develop it.

Package tests

twill comes with several unit tests. They depend on nose and Quixote 2.3. To run them, type 'python setup.py test' in the top package directory.

Licensing

twill 0.8 and above are licensed under the MIT license. All code currently contained in twill is Copyright (C) 2005, 2006, 2007 C. Titus Brown <titus@idyll.org>.

In plain English: I own the code, but you're welcome to use it, subsume it into other projects, and distribute it freely. However, you must retain copyright attribution.

pyparsing, BeautifulSoup, and mechanize are all included with twill, but are under their own licenses.

Developer releases

"Developer releases" incorporating all recent significant changes are made available at http://darcs.idyll.org/~t/projects/twill-latest.tar.gz

You can install them with easy_install by typing

easy_install -f http://darcs.idyll.org/~t/projects/twill-latest.tar.gz

Obtaining twill with darcs

darcs is a distributed version control system that allows users to maintain their own versions of various programs. It's a convenient way for me to make the latest source code available quickly, and it also is a dandy way to submit patches.

To obtain twill using darcs, install darcs and then type

darcs get http://darcs.idyll.org/~t/projects/twill/

To propose a change to the lead developer, make the changes and then do a 'send':

darcs record -am "explanation of change"
darcs send -a

To pull in changes made by the lead developer some time later:

darcs pull

Advanced documentation

twill uses a melange of different packages. Here are some potentially useful links:

Future Plans

TODO:

for 9.0 (beta).

  1. RECORDING.
  2. execute directories/directory trees.
  3. expose 'browser' & document re Grig; regexp esp.; make source docs.
  4. twill-fork: make file writing stuff optional; test massive fork fn.
  5. add config directives for socket timeout.
  6. directory stuff: import from specific place, access script location.

Misc fixes & cleanups:

  1. Paul McGuire's pyparsing suggestions.
  2. cookie "1" vs 1, in cookielib.
  3. extend_with etc. -- module namespaces, e.g. extmodule.fn?
  4. implement more complex proxy support. proxies proxies proxies.
  5. 'history' to show command history...
  6. record commands to script
  7. command line option for stdout -> file

Random ideas:

  1. Build twill interfaces for zope.testbrowser and mechanoid and pycurl.

    (This shouldn't be too difficult, since both are based on mechanize...)

  2. Develop twill-like languages for PAMIE and PyXPCOM. Note that the hard work, at least for PAMIE, is already done; check out the PAMIE examples page.

Contributions are welcome & will be duly acknowledged!

twill-0.9/doc/developer.txt0000644000175000001440000000722710735115574014212 0ustar tusers================ Developing twill ================ twill is entirely written in Python. You will need Python 2.3 or later to develop it. Package tests ~~~~~~~~~~~~~ twill comes with several unit tests. They depend on nose_ and `Quixote 2.3`_. To run them, type 'python setup.py test' in the top package directory. .. _nose: http://somethingaboutorange.com/mrl/projects/nose/ .. _Quixote 2.3: http://www.mems-exchange.org/software/quixote/ Licensing ~~~~~~~~~ twill 0.8 and above are licensed under the `MIT license`_. All code currently contained in twill is Copyright (C) 2005, 2006, 2007 C. Titus Brown . In plain English: I own the code, but you're welcome to use it, subsume it into other projects, and distribute it freely. However, you must retain copyright attribution. pyparsing_, BeautifulSoup_, and mechanize_ are all included with twill, but are under their own licenses. .. _MIT license: http://www.opensource.org/licenses/mit-license.php Developer releases ~~~~~~~~~~~~~~~~~~ "Developer releases" incorporating all recent significant changes are made available at http://darcs.idyll.org/~t/projects/twill-latest.tar.gz You can install them with easy_install by typing :: easy_install -f http://darcs.idyll.org/~t/projects/twill-latest.tar.gz Obtaining twill with darcs ~~~~~~~~~~~~~~~~~~~~~~~~~~ darcs_ is a distributed version control system that allows users to maintain their own versions of various programs. It's a convenient way for me to make the latest source code available quickly, and it also is a dandy way to submit patches. To obtain twill using darcs, install darcs and then type :: darcs get http://darcs.idyll.org/~t/projects/twill/ To propose a change to the lead developer, make the changes and then do a 'send': :: darcs record -am "explanation of change" darcs send -a To pull in changes made by the lead developer some time later: :: darcs pull Advanced documentation ~~~~~~~~~~~~~~~~~~~~~~ twill uses a melange of different packages. Here are some potentially useful links: * `urllib2.py: the missing manual`_ -- a detailed discussion of urllib2 functionality that can be directly applied to twill. .. _`urllib2.py: the missing manual`: http://www.voidspace.org.uk/python/articles/urllib2.shtml Future Plans ------------ TODO: for 9.0 (beta). 1. RECORDING. 2. execute directories/directory trees. 3. expose 'browser' & document re Grig; regexp esp.; make source docs. 4. twill-fork: make file writing stuff optional; test massive fork fn. 5. add config directives for socket timeout. 6. directory stuff: import from specific place, access script location. Misc fixes & cleanups: 1. Paul McGuire's pyparsing suggestions. 2. cookie "1" vs 1, in cookielib. 3. extend_with etc. -- module namespaces, e.g. extmodule.fn? 4. implement more complex proxy support. proxies proxies proxies. 5. 'history' to show command history... 6. record commands to script 7. command line option for stdout -> file Random ideas: 1. Build twill interfaces for zope.testbrowser and mechanoid and pycurl. (This shouldn't be too difficult, since both are based on mechanize...) 2. Develop twill-like languages for PAMIE_ and PyXPCOM_. Note that the hard work, at least for PAMIE, is already done; check out the `PAMIE examples page`_. Contributions are welcome & will be duly acknowledged! .. _mechanize: http://wwwsearch.sf.net/ .. _pyparsing: http://pyparsing.sourceforge.net/ .. _darcs: http://abridgegame.org/darcs/ .. _PAMIE: http://pamie.sourceforge.net/ .. _PAMIE examples page: tutoriala.html .. _PyXPCOM: http://kb.mozillazine.org/PyXPCOM .. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/ twill-0.9/doc/examples.html0000644000175000001440000000313510735115574014162 0ustar tusers twill Examples

twill Examples

Example: logging into slashdot.

This script logs you into slashdot (assuming you have an account!).

setlocal username <your username>
setlocal password <your password>

go http://www.slashdot.org/
formvalue 1 unickname $username
formvalue 1 upasswd $password
submit

code 200     # make sure form submission is correct!

Example: searching Google and going to the first hit

Please be aware that automated searching of Google violates their Terms of Service. This is for example purposes only!

setlocal query "twill Python"

go http://www.google.com/

fv 1 q $query
submit btnI     # use the "I'm feeling lucky" button

show
twill-0.9/doc/examples.txt0000644000175000001440000000146110735115574014035 0ustar tusers============== twill Examples ============== Example: logging into slashdot. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This script logs you into slashdot (assuming you have an account!). :: setlocal username setlocal password go http://www.slashdot.org/ formvalue 1 unickname $username formvalue 1 upasswd $password submit code 200 # make sure form submission is correct! Example: searching Google and going to the first hit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *Please be aware that automated searching of Google violates their Terms of Service. This is for example purposes only!* :: setlocal query "twill Python" go http://www.google.com/ fv 1 q $query submit btnI # use the "I'm feeling lucky" button show twill-0.9/doc/extensions.html0000644000175000001440000002536210735115574014551 0ustar tusers twill Extension Modules

twill Extension Modules

Several different extension modules are distributed with twill, under 'twill.extensions'.

match_parse -- extensions for slicing and dicing variables with regexps

The 'match_parse' extension module contains a number of functions for dealing with multiple matches, etc. Here's some example code:

extend_with match_parse
go http://www.idyll.org/

split "org"
echo __matchlist__

findall "t."
echo __matchlist__

split "org"
popmatch 0
getmatch test 'm[0].split()'
showvar test

split "org"
setmatch "m.split()[0]"

popmatch 0
echo __matchlist__
echo __match__

require -- assert that specific conditions hold after each page is loaded

The 'require' extension module contains four functions that let you assert specific conditions. For example,

extend_with require
require success

will assert that after each page load, the HTTP code is 200 ("success"). 'require' does nothing on a 'back' call (which doesn't actually reload a page).

Currently there are only two assertions available, 'success' and 'links_ok'. 'links_ok' automatically runs 'check_links' (see above) after each page is loaded. 'links_ok' will not check links twice unless you call 'flush_visited' (see below).

The other functions in this module are:

skip_require -- don't check the requirements for the next action.

no_require -- turn off requirements checking & reset conditions.

flush_visited -- flush the list of visited links.

mailman_sf -- discard spam messages from your SourceForge mailing lists

The 'mailman_sf' extension module contains two functions, 'discard_all_messages' and 'exit_if_empty'. Here's some example code:

# load in the mailman_sf extensions module
extend_with mailman_sf

# unfortunately we have to hard-code in the mailing list name for
# the moment.  not sure how to do substitutions here.
go https://lists.sourceforge.net/lists/admindb/pywx-announce

# fill out the page with the list password.
getpassword "Enter list password: "
formvalue 1 adminpw __password__
submit 0
code 200

# if there aren't any messages on the page, exit.
exit_if_empty

# if not empty, discard all messages & submit
discard_all_messages
submit 0

argparse -- pass arguments into twill scripts via sys.argv

The 'argparse' extension module contains one function, 'get_args'. 'get_args' loads all of the post-scriptfile command-line arguments into variables, e.g.

% ./twill-sh script1 script2 script3 -- val1 val2 val3
...
>> extend_with argparse
>> get_args
>> echo "args are ${arg1} ${args} ${arg3}"
args are val1 val2 val3

dns_check -- make assertions about domain name service records

dns_check implements functions to test domain name service (DNS) resolution. You'll need dnspython to use it.

Functions:

  • dns_resolves -- assert that a host resolves to a specific IP address.
  • dns_a -- assert that a host directly resolves to a specific IP address
  • dns_cname -- assert that a host is an alias for another hostname.
  • dnx_mx -- assert that a given host is a mail exchanger for the given name.
  • dns_ns -- assert that a given hostname is a name server for the given name.

csv_iterate -- iterate over lists of values

csv_iterate provides a single function, 'csv_iterate', that lets you run a script once for each row in a file of comma-separated values. For example, if you had a file 'test.csv' containing

value1, value2
value3, value4

and a script 'test' containing

echo "first value is ${col1}, second is ${col2}"

then

>> csv_iterate test.csv test

would produce

first value is value1, second is value2
first value is value3, second is value4

This is designed for building test suites containing many different combinations of values for specific forms.

dirstack -- manipulate the current working directory (cwd)

dirstack provides two functions, 'chdir' and 'popd', which change the directory and recover the original directory, respectively. The cwd is kept in the global variable '__dir__'.

For example

>> chdir /tmp
>> echo __dir__
>> popd
>> echo __dir__

will change to the '/tmp' directory, print out '/tmp', and then change back to the original directory.

formfill -- convenience functions for filling out forms

formfill provides three functions, 'fv_match', 'fv_multi', and 'fv_multi_sub'.

'fv_match' allows you to set many form fields all at once, based on regexp matching to the form field name, e.g.

fv_match <form> test-(\d+) 'hello world!'

will set all fields with the name 'test-#' (# is any number) to 'hello world!'

'fv_multi' allows you to set many form fields at once, using a more compact notation:

fv_multi <form> field1=value1 field2=value2 ...

'fv_multi' uses the same field-matching convention that 'formvalue' uses: it simply iterates over all arguments, splits at the first '=', and runs 'formvalue' directly.

'fv_multi_sub' does the same thing as 'fv_multi' and then executes 'submit'.

(Thanks to Ben Bangert for the idea!)

twill-0.9/doc/extensions.txt0000644000175000001440000001477010735115574014425 0ustar tusers======================= twill Extension Modules ======================= .. contents:: Several different extension modules are distributed with twill, under 'twill.extensions'. check_links -- a simple link checker ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A simple link checker is included with twill. To use it, do :: go extend_with check_links check_links 'check_links' will visit each link on the current page and verify that the HTTP status code is 200 (success). A list of failing links will be printed out at the end of the function. The 'check_links' function takes an optional argument, a regexp to filter for links that should be checked. By default the filter matches everything; if you only wanted to check links on e.g. idyll.org, you could do :: go http://idyll.org/ check_links .*\.idyll\.org A few notes: * 'follow' is used to visit each link, so the referrer URL should be set correctly. * HTTP basic authentication and cookies work as you'd expect, so you can set up twill with all of the permissions necessary to visit restricted links & only then run check_links. match_parse -- extensions for slicing and dicing variables with regexps ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The 'match_parse' extension module contains a number of functions for dealing with multiple matches, etc. Here's some example code: :: extend_with match_parse go http://www.idyll.org/ split "org" echo __matchlist__ findall "t." echo __matchlist__ split "org" popmatch 0 getmatch test 'm[0].split()' showvar test split "org" setmatch "m.split()[0]" popmatch 0 echo __matchlist__ echo __match__ require -- assert that specific conditions hold after each page is loaded ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The 'require' extension module contains four functions that let you assert specific conditions. For example, :: extend_with require require success will assert that after each page load, the HTTP code is 200 ("success"). 'require' does nothing on a 'back' call (which doesn't actually reload a page). Currently there are only two assertions available, 'success' and 'links_ok'. 'links_ok' automatically runs 'check_links' (see above) after each page is loaded. 'links_ok' will not check links twice unless you call 'flush_visited' (see below). The other functions in this module are: **skip_require** -- don't check the requirements for the next action. **no_require** -- turn off requirements checking & reset conditions. **flush_visited** -- flush the list of visited links. mailman_sf -- discard spam messages from your SourceForge mailing lists ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The 'mailman_sf' extension module contains two functions, 'discard_all_messages' and 'exit_if_empty'. Here's some example code: :: # load in the mailman_sf extensions module extend_with mailman_sf # unfortunately we have to hard-code in the mailing list name for # the moment. not sure how to do substitutions here. go https://lists.sourceforge.net/lists/admindb/pywx-announce # fill out the page with the list password. getpassword "Enter list password: " formvalue 1 adminpw __password__ submit 0 code 200 # if there aren't any messages on the page, exit. exit_if_empty # if not empty, discard all messages & submit discard_all_messages submit 0 argparse -- pass arguments into twill scripts via sys.argv ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The 'argparse' extension module contains one function, 'get_args'. 'get_args' loads all of the post-scriptfile command-line arguments into variables, e.g. :: % ./twill-sh script1 script2 script3 -- val1 val2 val3 ... >> extend_with argparse >> get_args >> echo "args are ${arg1} ${args} ${arg3}" args are val1 val2 val3 dns_check -- make assertions about domain name service records ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ dns_check implements functions to test domain name service (DNS) resolution. You'll need dnspython_ to use it. Functions: * dns_resolves -- assert that a host resolves to a specific IP address. * dns_a -- assert that a host directly resolves to a specific IP address * dns_cname -- assert that a host is an alias for another hostname. * dnx_mx -- assert that a given host is a mail exchanger for the given name. * dns_ns -- assert that a given hostname is a name server for the given name. .. _dnspython: http://www.dnspython.org/ csv_iterate -- iterate over lists of values ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ csv_iterate provides a single function, 'csv_iterate', that lets you run a script once for each row in a file of comma-separated values. For example, if you had a file 'test.csv' containing :: value1, value2 value3, value4 and a script 'test' containing :: echo "first value is ${col1}, second is ${col2}" then :: >> csv_iterate test.csv test would produce :: first value is value1, second is value2 first value is value3, second is value4 This is designed for building test suites containing many different combinations of values for specific forms. dirstack -- manipulate the current working directory (cwd) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ dirstack provides two functions, 'chdir' and 'popd', which change the directory and recover the original directory, respectively. The cwd is kept in the global variable '__dir__'. For example :: >> chdir /tmp >> echo __dir__ >> popd >> echo __dir__ will change to the '/tmp' directory, print out '/tmp', and then change back to the original directory. formfill -- convenience functions for filling out forms ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ formfill provides three functions, 'fv_match', 'fv_multi', and 'fv_multi_sub'. 'fv_match' allows you to set many form fields all at once, based on regexp matching to the form field name, e.g. :: fv_match
test-(\d+) 'hello world!' will set all fields with the name 'test-#' (# is any number) to 'hello world!' 'fv_multi' allows you to set many form fields at once, using a more compact notation: :: fv_multi field1=value1 field2=value2 ... 'fv_multi' uses the same field-matching convention that 'formvalue' uses: it simply iterates over all arguments, splits at the first '=', and runs 'formvalue' directly. 'fv_multi_sub' does the same thing as 'fv_multi' and then executes 'submit'. (Thanks to Ben Bangert for the idea!) twill-0.9/doc/index.html0000644000175000001440000002756110735115574013464 0ustar tusers twill: a simple scripting language for Web browsing

twill: a simple scripting language for Web browsing

twill is a simple language that allows users to browse the Web from a command-line interface. With twill, you can navigate through Web sites that use forms, cookies, and most standard Web features.

twill supports automated Web testing and has a simple Python interface. Check out the examples!

twill is open source and written in Python.

Downloading twill

The latest release of twill is twill 0.9, released Thursday, December 27th, 2007; it is available for download at http://darcs.idyll.org/~t/projects/twill-0.9.tar.gz. You can also use Python's easy_install to install or upgrade twill.

twill works with Python 2.3 or later.

To start using twill, install it and then type twill-sh. At the prompt type:

go http://www.slashdot.org/
show
showforms
showhistory

Documentation

The documentation for the latest release is always at http://twill.idyll.org/.

The in-development version of the docs can be found at http://darcs.idyll.org/~t/projects/twill/doc/.

Documentation is available for the following topics:

Mailing List

There's a moderately active mailing list at twill@lists.idyll.org; there is also an archive for this list.

Contributing

Bug reports, fixes, extensions, and links to interesting blog or use case reports are always welcome! You can send them to the mailing list or to Titus directly.

When reporting bugs, please be sure to use the '-f' flag for twill-sh so that we can see the full traceback.

Authors and License

The main author of twill is C. Titus Brown, titus@idyll.org. A number of people have contributed bug reports and code since the first release; they are acknowledged below.

The twill source code is Copyright (C) 2005, 2006, 2007 C. Titus Brown. twill is available under the MIT license.

pyparsing, mechanize, and BeautifulSoup are included with twill for convenience, but are under their own licenses.

Acknowledgements

Cory Dodt had a great idea with PBP, and I thank him for his insight. Ian Bicking gave me the idea of reimplementing PBP on top of IPython (since abandoned in favor of cmd), and suggested the "in-process" hack. Grig Gheorghiu was strangely enthusiastic about the simple demo I showed him and has religiously promoted twill ever since. John J. Lee has promptly and enthusiastically checked in my various patches to mechanize. Michele Simionato is an early adopter who has helped quite a bit. Thanks, guys...

Bug reports have come in from the following fine people: Chris Miles, MATSUNO Tokuhiro, Elvelind Grandin, Mike Rovner, sureshvv, Terry Peppers, Kieran Holland, Alexander Shvedunov, Norman Khine, Leonardo Santagada, Sebastien Pierre, Herve Cauwelier, aledain, Uy Do, David Hancock, and Tomi Hautakoski.

Patches have been submitted by Joeri van Ruth, Paul McGuire, Ed Rahn, Nic Ferrier, Robert Leftwich, James Cameron, William Volkman, Tommi Virtanen, Simon Buenzli, sureshvv, Jeff Martin, Stephen Thorne, and Bob Halley.

Features were proposed by Ben Bangert, and Tristan De Buysscher.

Thanks, all!

twill-0.9/doc/index.txt0000644000175000001440000001117510735115574013331 0ustar tusers=================================================== twill: a simple scripting language for Web browsing =================================================== twill is a simple language that allows users to browse the Web from a command-line interface. With twill, you can navigate through Web sites that use forms, cookies, and most standard Web features. twill supports automated Web testing and has a simple Python interface. Check out the examples_! twill is open source and written in Python. Downloading twill ----------------- The latest release of twill is `twill 0.9 `__, released Thursday, December 27th, 2007; it is available for download at http://darcs.idyll.org/~t/projects/twill-0.9.tar.gz. You can also use Python's easy_install_ to install or upgrade twill. twill works with Python 2.3 or later. To start using twill, install it and then type ``twill-sh``. At the prompt type: :: go http://www.slashdot.org/ show showforms showhistory Documentation ------------- The documentation for the latest release is always at http://twill.idyll.org/. The in-development version of the docs can be found at http://darcs.idyll.org/~t/projects/twill/doc/. Documentation is available for the following topics: * `Examples`_ -- some short examples. * `Web browsing with twill`_ -- General introduction to twill. * `Language reference`_ -- the twill scripting language. * `Testing Web sites with twill`_ -- how to use twill to test Web sites. * `Extension modules`_ -- extension modules that come with twill. * `Python API`_ -- for Python programmers interested in using twill from Python. * `Developer information`_ -- for Python developers interested in extending or fixing twill. * `Other tools and packages`_ -- projects relevant to, or based upon, twill. * `Known problems`_ -- those problems that are still lurking in the code. Mailing List ------------ There's a moderately active mailing list at twill@lists.idyll.org; there is also an archive_ for this list. Contributing ------------ Bug reports, fixes, extensions, and links to interesting blog or use case reports are always welcome! You can send them to the mailing list or to Titus directly. When reporting bugs, please be sure to use the '-f' flag for ``twill-sh`` so that we can see the full traceback. Authors and License ------------------- The main author of twill is C. Titus Brown, titus@idyll.org. A number of people have contributed bug reports and code since the first release; they are acknowledged below. The twill source code is Copyright (C) 2005, 2006, 2007 C. Titus Brown. twill is available under the `MIT license`_. pyparsing_, mechanize_, and BeautifulSoup_ are included with twill for convenience, but are under their own licenses. Acknowledgements ---------------- Cory Dodt had a great idea with PBP, and I thank him for his insight. Ian Bicking gave me the idea of reimplementing PBP on top of IPython (since abandoned in favor of cmd_), and suggested the "in-process" hack. Grig Gheorghiu was strangely enthusiastic about the simple demo I showed him and has religiously promoted twill ever since. John J. Lee has promptly and enthusiastically checked in my various patches to mechanize. Michele Simionato is an early adopter who has helped quite a bit. Thanks, guys... Bug reports have come in from the following fine people: Chris Miles, MATSUNO Tokuhiro, Elvelind Grandin, Mike Rovner, sureshvv, Terry Peppers, Kieran Holland, Alexander Shvedunov, Norman Khine, Leonardo Santagada, Sebastien Pierre, Herve Cauwelier, aledain, Uy Do, David Hancock, and Tomi Hautakoski. Patches have been submitted by Joeri van Ruth, Paul McGuire, Ed Rahn, Nic Ferrier, Robert Leftwich, James Cameron, William Volkman, Tommi Virtanen, Simon Buenzli, sureshvv, Jeff Martin, Stephen Thorne, and Bob Halley. Features were proposed by Ben Bangert, and Tristan De Buysscher. Thanks, all! ---- .. _Installing twill: install.html .. _Web browsing with twill: browsing.html .. _Language reference: commands.html .. _Developer information: developer.html .. _Python API: python-api.html .. _Testing Web sites with twill: testing.html .. _Other tools and packages: other.html .. _Known problems: problems.html .. _Extension modules: extensions.html .. _archive: http://lists.idyll.org/pipermail/twill/ .. _cmd: http://docs.python.org/lib/module-cmd.html .. _MIT license: http://www.opensource.org/licenses/mit-license.php .. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall .. _mechanize: http://wwwsearch.sf.net/ .. _pyparsing: http://pyparsing.sourceforge.net/ .. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/ .. _examples: examples.html twill-0.9/doc/install.html0000644000175000001440000001155610735115574014020 0ustar tusers Installing twill

Installing twill

The recommended way to install twill is to use easy_install:

easy_install twill

You can also install it directly from the distribution tar.gz file by unpacking the .tar.gz file and running

python setup.py install

Either way, you need to have setuptools installed; setuptools doesn't come with Python 2.3 or Python 2.4, so you will need to install it before you install twill. To install both easy_install and setuptools, grab ez_setup.py from

http://peak.telecommunity.com/dist/ez_setup.py

and run 'python ez_setup.py'. This will install setuptools and put an easy_install program in your Python scripts directory.

Installing twill on Windows

The most frequent problem people seem to encounter on Windows is the inability to run 'twill-sh'. This happens because the Python installation's 'scripts' directory isn't in the path by default. According to the Python Windows FAQ, the magic incantation is something like this:

PATH C:\Python23\scripts;%PATH%

(You should replace "C:Python23" with the directory of your Python installation.)

Installing twill on Mac OS X

As with Windows, the problem most frequently encountered seems to be that the Python 'bin' directory isn't part of your path by default. This means that the twill shell program, 'twill-sh', will not be found unless you add the script directory into your path. On current versions of Mac OS X, something like this should work: for bash shells,:

export PATH=$PATH:/System/Library/Frameworks/Python.framework/Versions/2.3/bin

or for csh-based shells,

set path=($path /System/Library/Frameworks/Python.framework/Versions/2.3/bin)

Troubleshooting your installation

The first and only thing you should do before asking the list (twill@lists.idyll.org) for help is to see if you have the twill package installed correctly:

% python
>>> import twill.shell
>>> twill.shell.main()

(This should drop you into the twill shell irrespective of whether 'twill-sh' is on your path.) If this works, twill is installed correctly and you just need to adjust your path (see above for examples). If you still need help, or the above code doesn't work, please copy and paste the results of entering the above code into your e-mail to the list -- thanks!

Upgrading twill

If you don't want to download a new tar.gz file, you can use easy_install to upgrade twill. To get the latest release, use

easy_install -U twill

To download the latest development release (which is usually pretty stable) use

easy_install -f http://issola.caltech.edu/~t/dist/ -U twill
twill-0.9/doc/install.txt0000644000175000001440000000564010735115574013670 0ustar tusers================ Installing twill ================ .. contents:: The recommended way to install twill is to use easy_install_: :: easy_install twill You can also install it directly from the distribution tar.gz file by unpacking the .tar.gz file and running :: python setup.py install Either way, **you need to have setuptools installed**; setuptools doesn't come with Python 2.3 or Python 2.4, so you will need to install it before you install twill. To install both easy_install and setuptools, grab ez_setup.py from :: http://peak.telecommunity.com/dist/ez_setup.py and run 'python ez_setup.py'. This will install setuptools and put an easy_install program in your Python scripts directory. Installing twill on Windows ~~~~~~~~~~~~~~~~~~~~~~~~~~~ The most frequent problem people seem to encounter on Windows is the inability to run 'twill-sh'. This happens because the Python installation's 'scripts' directory isn't in the path by default. According to the `Python Windows FAQ`_, the magic incantation is something like this: :: PATH C:\Python23\scripts;%PATH% (You should replace "C:\Python23" with the directory of your Python installation.) Installing twill on Mac OS X ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ As with Windows, the problem most frequently encountered seems to be that the Python 'bin' directory isn't part of your path by default. This means that the twill shell program, 'twill-sh', will not be found unless you add the script directory into your path. On current versions of Mac OS X, something like this should work: for bash shells,:: export PATH=$PATH:/System/Library/Frameworks/Python.framework/Versions/2.3/bin or for csh-based shells, :: set path=($path /System/Library/Frameworks/Python.framework/Versions/2.3/bin) Troubleshooting your installation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The first and only thing you should do before asking the list (twill@lists.idyll.org) for help is to see if you have the twill package installed correctly: :: % python >>> import twill.shell >>> twill.shell.main() (This should drop you into the twill shell irrespective of whether 'twill-sh' is on your path.) **If this works**, twill is installed correctly and you just need to adjust your path (see above for examples). If you still need help, or the above code doesn't work, please copy and paste the results of entering the above code into your e-mail to the list -- thanks! Upgrading twill ~~~~~~~~~~~~~~~ If you don't want to download a new tar.gz file, you can use easy_install to upgrade twill. To get the latest release, use :: easy_install -U twill To download the latest development release (which is usually pretty stable) use :: easy_install -f http://issola.caltech.edu/~t/dist/ -U twill .. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall .. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools .. _Python Windows FAQ: http://www.python.org/doc/faq/windows.htmltwill-0.9/doc/make-epydoc.sh0000644000175000001440000000014310735115574014204 0ustar tusers#! /bin/bash epydoc --docformat restructuredtext --no-private -n twill -o ./epydoc-html/ ../twill/ twill-0.9/doc/make-quick-fix.sh0000644000175000001440000000010110735115574014613 0ustar tusers#! /usr/bin/env bash darcs dist -d twill-latest-`date +%Y-%m-%d` twill-0.9/doc/other.html0000644000175000001440000001335710735115574013474 0ustar tusers Propaganda, tools and packages

Propaganda, tools and packages

Other opinions

Grig Gheorgiu has written a blog entry on Web app testing using twill. Michele Simionato wrote a nice long article on Testing Web Apps, and Nitesh Djanjani tried it out as well.

Tools based on twill

Petrik De Heus wrote a Wikipedia watchlist RSS feed with twill.

Offshoots from the twill project

I wrote a simple wsgi_intercept standalone package based on the WSGI interception stuff for in-process testing of WSGI apps by twill. This is for people who want to talk directly to their Web apps without going through a network connection.

Other Python-based Web testing and browsing tools

For those who want to use Python-based toolkits to test their Web apps, there are many great options. Here are the ones I know about:

PBP is very similar to twill -- twill was initially based on it -- but the project seems to be more-or-less defunct. Both twill and PBP are based on mechanize.

funkload is a nifty looking tool that does functional load testing. It is built on webunit.

webtest is an extension to unittest for testing Web frameworks. It's primarily used for CherryPy, I think.

zope.testbrowser is a wrapper around mechanize that exposes a more convenient interface for testing. Check out the zope.testbrowser README for more information.

mechanoid is a fork of mechanize that claims many bug fixes and a different programming style. It's primarily used for scripting Web sites, not for testing, but it can easily be used for testing.

PAMIE and PyXPCOM provide a Python interface for interacting with IE and Mozilla-based browsers, respectively.

Finally, Selenium is an in-browser testing system that several people have given rave reviews. Check out this article on Selenium for some starting tips. Note that it's not written in Python...

You might also consider checking out Ian Bicking's proto-implementation of twill in Javascript.

(Thanks to Grig Gheorghiu for assembling much of this list!)

Tools in Other Languages

As twill is based upon mechanize, so is Perl's WWW::Mechanize::Shell based upon WWW::Mechanize. There's even an HTTP::Recorder. WebTst also looks interesting.

For PHP, web_tester looks useful.

twill-0.9/doc/other.txt0000644000175000001440000001133510735115574013341 0ustar tusers============================== Propaganda, tools and packages ============================== Other opinions ~~~~~~~~~~~~~~ Grig Gheorgiu has written a blog entry on `Web app testing using twill`_. Michele Simionato wrote a nice long article on `Testing Web Apps`_, and Nitesh Djanjani `tried it out`_ as well. .. _Web app testing using twill: http://agiletesting.blogspot.com/2005/09/web-app-testing-with-python-part-3.html .. _Testing Web Apps: http://www.onlamp.com/pub/a/python/2005/11/03/twill.html .. _tried it out: http://www.oreillynet.com/pub/wlg/8201 Tools based on twill ~~~~~~~~~~~~~~~~~~~~ Petrik De Heus wrote a Wikipedia `watchlist RSS feed`_ with twill. .. _watchlist RSS feed: http://www.deheus.net/petrik/blog/post/68/ Tools useful for twill users ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Matt Harrison is working on a `TestGen4Web-to-twill converter`_. .. _TestGen4Web-to-twill converter: http://panela.blog-city.com/generate_twill_scripts_and_mechanize_unittests_from_testgen4.htm Offshoots from the twill project ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I wrote a simple `wsgi_intercept`_ standalone package based on the WSGI interception stuff for in-process testing of WSGI apps by twill. This is for people who want to talk directly to their Web apps without going through a network connection. .. _wsgi_intercept: http://darcs.idyll.org/~t/projects/wsgi_intercept/README.html Other Python-based Web testing and browsing tools ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For those who want to use Python-based toolkits to test their Web apps, there are many great options. Here are the ones I know about: PBP_ is very similar to twill -- twill was initially based on it -- but the project seems to be more-or-less defunct. Both twill and PBP are based on mechanize_. funkload_ is a nifty looking tool that does functional load testing. It is built on webunit_. webtest_ is an extension to ``unittest`` for testing Web frameworks. It's primarily used for CherryPy, I think. zope.testbrowser_ is a wrapper around mechanize_ that exposes a more convenient interface for testing. Check out the `zope.testbrowser README`_ for more information. mechanoid_ is a fork of mechanize_ that claims many bug fixes and a different programming style. It's primarily used for scripting Web sites, not for testing, but it can easily be used for testing. PAMIE_ and PyXPCOM_ provide a Python interface for interacting with IE and Mozilla-based browsers, respectively. Finally, Selenium_ is an in-browser testing system that several people have given rave reviews. Check out this `article on Selenium`_ for some starting tips. Note that it's not written in Python... You might also consider checking out Ian Bicking's proto-implementation of `twill in Javascript`_. (Thanks to Grig Gheorghiu for assembling much of this list!) Tools in Other Languages ~~~~~~~~~~~~~~~~~~~~~~~~ As twill is based upon mechanize, so is Perl's `WWW::Mechanize::Shell`_ based upon `WWW::Mechanize`_. There's even an `HTTP::Recorder`_. WebTst_ also looks interesting. For PHP, web_tester_ looks useful. .. _PBP: http://pbp.berlios.de/ .. _maxq: http://maxq.tigris.org/ .. _mechanize: http://wwwsearch.sf.net/ .. _pyparsing: http://pyparsing.sourceforge.net/ .. _funkload: http://funkload.nuxeo.org/ .. _webunit: http://mechanicalcat.net/tech/webunit/ .. _webtest: http://www.cherrypy.org/file/trunk/cherrypy/test/webtest.py .. _zope.testbrowser: http://cheeseshop.python.org/pypi?:action=display&name=zope.testbrowser .. _zope.testbrowser README: http://svn.zope.org/*checkout*/Zope3/trunk/src/zope/testbrowser/README.txt .. _Selenium: http://confluence.public.thoughtworks.org/display/SEL/Home .. _article on Selenium: http://agiletesting.blogspot.com/2005/10/article-on-selenium-in-october-issue.html .. _WSGI applications: http://www.python.org/peps/pep-0333.html .. _WSGI: http://www.python.org/peps/pep-0333.html .. _Quixote: http://www.mems-exchange.org/software/quixote/ .. _mechanoid: http://python.org/pypi/mechanoid .. _twill in Javascript: http://blog.ianbicking.org/twill-in-javascript.html .. _PAMIE: http://pamie.sourceforge.net/ .. _PAMIE examples page: tutoriala.html .. _PyXPCOM: http://kb.mozillazine.org/PyXPCOM .. _tidy page: http://tidy.sourceforge.net/ .. _easy_install/setuptools: http://peak.telecommunity.com/DevCenter/EasyInstall .. _installing easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install .. _WebTst: http://webtst.sourceforge.net/documentation.html .. _`WWW::Mechanize`: http://search.cpan.org/perldoc?WWW::Mechanize::Shell .. _`WWW::Mechanize::Shell`: http://search.cpan.org/perldoc?WWW::Mechanize::Shell .. _`HTTP::Recorder`: http://www.perl.com/pub/a/2004/06/04/recorder.html .. _web_tester: http://www.lastcraft.com/web_tester_documentation.phptwill-0.9/doc/problems.html0000644000175000001440000001052210735115574014165 0ustar tusers Known Problems or Error Messages

Known Problems or Error Messages

Problems posted to the twill mailing list (or sent to Titus personally) will end up here if they are either outside the scope of twill, or intractable within the context of the twill project!

Last updated Apr 2006.

1. inspect module masking

The error message

AttributeError: 'module' object has no attribute 'getmro'

is caused by masking of the default Python 'inspect' module with another module named 'inspect'. Known to happen in Webware installs. (Reported by David Hancock.)

2. Empty 'textarea' form elements are filled with a newline by 'tidy'.

Specifically,

<textarea></textarea>

is turned into

<textarea>
</textarea>

by tidy. Set 'config use_tidy 0' to turn this off. You can use

<textarea />

to sneak an empty textarea through tidy.

(Reported by Gabor Farkas.)

3. Unicode support is still young.

It's not yet clear how well mechanize and twill adhere to either the official recommendations (http://www.w3.org/TR/REC-html40/charset.html) or the unofficial recommendations (http://www.alanwood.net/unicode/htmlunicode.html) for dealing with unicode. There are definitely some problems in the version of mechanize included with twill in 0.8.4; this should be resolved in later releases.

4. Various kinds of broken HTML will break the form parser.

For example, twill (or, rather, ClientForm) sees no forms in this HTML

<font>
<INPUT type="this">

<FORM>
<input type="blah">
</form>

5. Windows problems and tidy

If you get the error message

The process cannot access the file because it is being used by another process.

on Windows, try turning 'tidy' processing off:

config use_tidy 0

(and report it to the list if this does or does not solve the problem!)

6. sys.path mangling

twill, on import, alters the Python sys.path by appending twill/extensions to sys.path and inserting twill/other_packages in the front of the module path. This may cause problems with packages with conflicting names.

In particular, other modules whose names "shadow" twill's included extensions will be imported instead of the included extensions. Also, only twill's versions of pyparsing, mechanize, ClientForm, and ClientCookie will be import-able.

There is no good, simple way to get around this; setuptools may be our only hope.

twill-0.9/doc/problems.txt0000644000175000001440000000514410735115574014044 0ustar tusers================================ Known Problems or Error Messages ================================ Problems posted to the twill mailing list (or sent to Titus personally) will end up here if they are either outside the scope of twill, or intractable within the context of the twill project! Last updated Apr 2006. 1. inspect module masking ~~~~~~~~~~~~~~~~~~~~~~~~~ The error message :: AttributeError: 'module' object has no attribute 'getmro' is caused by masking of the default Python 'inspect' module with another module named 'inspect'. Known to happen in Webware installs. (Reported by David Hancock.) 2. Empty 'textarea' form elements are filled with a newline by 'tidy'. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Specifically, :: is turned into :: by tidy. Set 'config use_tidy 0' to turn this off. You can use ::