cacheimages Namespace Reference

Functions
	convertImgs (infile)
	Flags ()
	main ()

Variables
	SWARM_PATH = dirname(abspath(__file__))
	CLIENT_PATH = dirname(dirname(SWARM_PATH))
	CLIENT_TOOLS_PATH = join(CLIENT_PATH, 'tools')
str	converter = CLIENT_TOOLS_PATH + '/htmlconverter.py'

Detailed Description

This script finds all HTML pages in a folder and downloads all images, replacing
the urls with local ones.

Function Documentation

convertImgs()

cacheimages.convertImgs

(

infile

)

Definition at line 23 of file cacheimages.py.

def convertImgs(infile):
    global options
    try:
        htmlconverter.convertForOffline(infile,
                                        infile,
                                        verbose=options.verbose,
                                        encode_images=options.inline_images)
        print('Converted ' + infile)
    except BaseException as e:
        print('Caught error: %s' % e)
 
 

Flags()

cacheimages.Flags

(

)

 Constructs a parser for extracting flags from the command line. 

Definition at line 35 of file cacheimages.py.

def Flags():
    """ Constructs a parser for extracting flags from the command line. """
    parser = optparse.OptionParser()
    parser.add_option(
        "--inline_images",
        help=("Encode img payloads as data:// URLs rather than local files."),
        default=False,
        action='store_true')
    parser.add_option("--verbose",
                      help="Print verbose output",
                      default=False,
                      action="store_true")
    return parser
 
 

main()

cacheimages.main

(

)

Definition at line 50 of file cacheimages.py.

def main():
    global options
    parser = Flags()
    options, args = parser.parse_args()
    print("args: %s" % args)
    if len(args) < 1 or 'help' in args[0]:
        print('Usage: %s DIRECTORY' % basename(sys.argv[0]))
        return 1
 
    dirname = args[0]
    print('Searching directory ' + dirname)
 
    files = []
    for root, dirs, fnames in os.walk(dirname):
        for fname in fnames:
            if fname.endswith('.html'):
                files.append(join(root, fname))
 
    count = 4 * multiprocessing.cpu_count()
    pool = multiprocessing.Pool(processes=count)
    # Note: need a timeout to get keyboard interrupt due to a Python bug
    pool.map_async(convertImgs, files).get(3600)  # one hour
 
 

Variable Documentation

CLIENT_PATH

cacheimages.CLIENT_PATH = dirname(dirname(SWARM_PATH))

Definition at line 13 of file cacheimages.py.

CLIENT_TOOLS_PATH

cacheimages.CLIENT_TOOLS_PATH = join(CLIENT_PATH, 'tools')

Definition at line 14 of file cacheimages.py.

converter

str cacheimages.converter = CLIENT_TOOLS_PATH + '/htmlconverter.py'

Definition at line 19 of file cacheimages.py.

SWARM_PATH

cacheimages.SWARM_PATH = dirname(abspath(__file__))

Definition at line 12 of file cacheimages.py.