<h2>org.das2.util.filesystem.HtmlUtil</h2><p>HTML utilities, such as getting a directory listing, where a "file" is a link
 below the directory we are listing, and read a URL into a String.</p>
<h2>HtmlUtil( )</h2>
<p></p>

<hr>
<a name="checkRedirect"></a>
<h2>checkRedirect</h2>
checkRedirect( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URLConnection.html'>java.net.URLConnection</a> urlConnection ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URLConnection.html'>java.net.URLConnection</a>

<p>check for 301, 302 or 303 redirects, and return a new connection in this case.
 This should be called immediately before the urlConnection.connect call,
 as this must connect to get the response code.</p>

<h3>Parameters</h3>
urlConnection - if an HttpUrlConnection, check for 301 or 302; return connection otherwise.

<h3>Returns:</h3>
a connection, typically the same one as passed in.
<h3>See Also:</h3>
<a href='null'>HttpUtil#checkRedirect(java.net.URLConnection)</a> <br>
<br><br>
<a href="https://github.com/autoplot/dev/search?q=checkRedirect&unscoped_q=checkRedirect">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#checkRedirect">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="consumeStream"></a>
<h2>consumeStream</h2>
consumeStream( <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> err ) &rarr; void

<p>nice clients consume both the stderr and stdout coming from websites.
 This reads everything off of the stream and closes it.
 http://docs.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html suggests that you "do not abandon connection"</p>

<h3>Parameters</h3>
err - the input stream

<h3>Returns:</h3>
void (returns nothing)

<h3>See Also:</h3>
<a href='null'>HttpUtil#consumeStream(java.io.InputStream)</a> <br>
<br><br>
<a href="https://github.com/autoplot/dev/search?q=consumeStream&unscoped_q=consumeStream">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#consumeStream">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="getDirectoryListing"></a>
<h2>getDirectoryListing</h2>
getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> urlStream ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a>

<p>Get the listing of the web directory, returning links that are "under" the given URL.
 Note this does not handle off-line modes where we need to log into
 a website first, as is often the case for a hotel.

 This was refactored to support caching of listings by simply writing the content to disk.</p>

<h3>Parameters</h3>
url - the address.
<br>
urlStream - stream containing the URL content, which must be UTF-8 (or US-ASCII)

<h3>Returns:</h3>
list of URIs referred to in the page.
<br><br>
<a href="https://github.com/autoplot/dev/search?q=getDirectoryListing&unscoped_q=getDirectoryListing">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getDirectoryListing">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> urlStream, boolean childCheck ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a><br>
getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a><br>
<hr>
<a name="getInputStream"></a>
<h2>getInputStream</h2>
getInputStream( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a>

<p>get the inputStream, following redirects if a 301 or 302 is encountered.  
 The scientist may be prompted for a password, but only if "user@" is
 in the URL.
 
 Note this does not explicitly close the connections
 to the server, and Java may not know to release the resources.  
 TODO: fix this by wrapping the input stream and closing the connection
 when the stream is closed.  This was done in Autoplot's DataSetURI.downloadResourceAsTempFile</p>

<h3>Parameters</h3>
url - an URL

<h3>Returns:</h3>
input stream
<h3>See Also:</h3>
<a href='http://www-pw.physics.uiowa.edu/~jbf/autoplot/doc/org/autoplot/datasource/DataSetURI.html#downloadResourceAsTempFile'>org.autoplot.datasource.DataSetURI#downloadResourceAsTempFile</a> <br>
<br><br>
<a href="https://github.com/autoplot/dev/search?q=getInputStream&unscoped_q=getInputStream">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getInputStream">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="getLinks"></a>
<h2>getLinks</h2>
getLinks( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/lang/String.html'>String</a> content ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/util/List.html'>java.util.List</a>

<p>return the links found in the content, using url as the context.</p>

<h3>Parameters</h3>
url - null or the url for the context.
<br>
content - the html content.

<h3>Returns:</h3>
a list of URLs.
<br><br>
<a href="https://github.com/autoplot/dev/search?q=getLinks&unscoped_q=getLinks">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getLinks">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="getMetadata"></a>
<h2>getMetadata</h2>
getMetadata( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/util/Map.html'>java.util.Map</a> props ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/util/Map.html'>java.util.Map</a>

<p>return the metadata about a URL.  This will support http, https,
 and ftp, and will check for redirects.  This will 
 allow caching of head requests.</p>

<h3>Parameters</h3>
url - ftp,https, or http URL
<br>
props - a java.util.Map

<h3>Returns:</h3>
the metadata
<h3>See Also:</h3>
<a href='null'>HttpUtil#getMetadata(java.net.URL, java.util.Map)</a> <br>
<br><br>
<a href="https://github.com/autoplot/dev/search?q=getMetadata&unscoped_q=getMetadata">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getMetadata">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="isDirectory"></a>
<h2>isDirectory</h2>
isDirectory( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) &rarr; boolean

<p></p>

<h3>Parameters</h3>
url - an URL

<h3>Returns:</h3>
boolean

<br><br>
<a href="https://github.com/autoplot/dev/search?q=isDirectory&unscoped_q=isDirectory">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#isDirectory">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>
<hr>
<a name="readToString"></a>
<h2>readToString</h2>
readToString( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) &rarr; <a href='https://docs.oracle.com/javase/8/docs/api/java/lang/String.html'>String</a>

<p>read the contents of the URL into a string, assuming UTF-8 encoding.</p>

<h3>Parameters</h3>
url - an URL

<h3>Returns:</h3>
a String

<br><br>
<a href="https://github.com/autoplot/dev/search?q=readToString&unscoped_q=readToString">[search for examples]</a>
 <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a>
 <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#readToString">[view on old javadoc]</a>
 <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a>
<br>
<br>