<h2>org.das2.util.filesystem.HtmlUtil</h2><p>HTML utilities, such as getting a directory listing, where a "file" is a link below the directory we are listing, and read a URL into a String.</p> <h2>HtmlUtil( )</h2> <p></p> <hr> <a name="checkRedirect"></a> <h2>checkRedirect</h2> checkRedirect( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URLConnection.html'>java.net.URLConnection</a> urlConnection ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URLConnection.html'>java.net.URLConnection</a> <p>check for 301, 302 or 303 redirects, and return a new connection in this case. This should be called immediately before the urlConnection.connect call, as this must connect to get the response code.</p> <h3>Parameters</h3> urlConnection - if an HttpUrlConnection, check for 301 or 302; return connection otherwise. <h3>Returns:</h3> a connection, typically the same one as passed in. <h3>See Also:</h3> <a href='null'>HttpUtil#checkRedirect(java.net.URLConnection)</a> <br> <br><br> <a href="https://github.com/autoplot/dev/search?q=checkRedirect&unscoped_q=checkRedirect">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#checkRedirect">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="consumeStream"></a> <h2>consumeStream</h2> consumeStream( <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> err ) → void <p>nice clients consume both the stderr and stdout coming from websites. This reads everything off of the stream and closes it. http://docs.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html suggests that you "do not abandon connection"</p> <h3>Parameters</h3> err - the input stream <h3>Returns:</h3> void (returns nothing) <h3>See Also:</h3> <a href='null'>HttpUtil#consumeStream(java.io.InputStream)</a> <br> <br><br> <a href="https://github.com/autoplot/dev/search?q=consumeStream&unscoped_q=consumeStream">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#consumeStream">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="getDirectoryListing"></a> <h2>getDirectoryListing</h2> getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> urlStream ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a> <p>Get the listing of the web directory, returning links that are "under" the given URL. Note this does not handle off-line modes where we need to log into a website first, as is often the case for a hotel. This was refactored to support caching of listings by simply writing the content to disk.</p> <h3>Parameters</h3> url - the address. <br> urlStream - stream containing the URL content, which must be UTF-8 (or US-ASCII) <h3>Returns:</h3> list of URIs referred to in the page. <br><br> <a href="https://github.com/autoplot/dev/search?q=getDirectoryListing&unscoped_q=getDirectoryListing">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getDirectoryListing">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> urlStream, boolean childCheck ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a><br> getDirectoryListing( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL[]</a><br> <hr> <a name="getInputStream"></a> <h2>getInputStream</h2> getInputStream( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html'>java.io.InputStream</a> <p>get the inputStream, following redirects if a 301 or 302 is encountered. The scientist may be prompted for a password, but only if "user@" is in the URL. Note this does not explicitly close the connections to the server, and Java may not know to release the resources. TODO: fix this by wrapping the input stream and closing the connection when the stream is closed. This was done in Autoplot's DataSetURI.downloadResourceAsTempFile</p> <h3>Parameters</h3> url - an URL <h3>Returns:</h3> input stream <h3>See Also:</h3> <a href='http://www-pw.physics.uiowa.edu/~jbf/autoplot/doc/org/autoplot/datasource/DataSetURI.html#downloadResourceAsTempFile'>org.autoplot.datasource.DataSetURI#downloadResourceAsTempFile</a> <br> <br><br> <a href="https://github.com/autoplot/dev/search?q=getInputStream&unscoped_q=getInputStream">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getInputStream">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="getLinks"></a> <h2>getLinks</h2> getLinks( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/lang/String.html'>String</a> content ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/util/List.html'>java.util.List</a> <p>return the links found in the content, using url as the context.</p> <h3>Parameters</h3> url - null or the url for the context. <br> content - the html content. <h3>Returns:</h3> a list of URLs. <br><br> <a href="https://github.com/autoplot/dev/search?q=getLinks&unscoped_q=getLinks">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getLinks">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="getMetadata"></a> <h2>getMetadata</h2> getMetadata( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url, <a href='https://docs.oracle.com/javase/8/docs/api/java/util/Map.html'>java.util.Map</a> props ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/util/Map.html'>java.util.Map</a> <p>return the metadata about a URL. This will support http, https, and ftp, and will check for redirects. This will allow caching of head requests.</p> <h3>Parameters</h3> url - ftp,https, or http URL <br> props - a java.util.Map <h3>Returns:</h3> the metadata <h3>See Also:</h3> <a href='null'>HttpUtil#getMetadata(java.net.URL, java.util.Map)</a> <br> <br><br> <a href="https://github.com/autoplot/dev/search?q=getMetadata&unscoped_q=getMetadata">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#getMetadata">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="isDirectory"></a> <h2>isDirectory</h2> isDirectory( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) → boolean <p></p> <h3>Parameters</h3> url - an URL <h3>Returns:</h3> boolean <br><br> <a href="https://github.com/autoplot/dev/search?q=isDirectory&unscoped_q=isDirectory">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#isDirectory">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br> <hr> <a name="readToString"></a> <h2>readToString</h2> readToString( <a href='https://docs.oracle.com/javase/8/docs/api/java/net/URL.html'>java.net.URL</a> url ) → <a href='https://docs.oracle.com/javase/8/docs/api/java/lang/String.html'>String</a> <p>read the contents of the URL into a string, assuming UTF-8 encoding.</p> <h3>Parameters</h3> url - an URL <h3>Returns:</h3> a String <br><br> <a href="https://github.com/autoplot/dev/search?q=readToString&unscoped_q=readToString">[search for examples]</a> <a href="https://github.com/autoplot/documentation/tree/master/javadoc/org/das2/util/filesystem/HtmlUtil.md">[view on GitHub]</a> <a href="http://www-pw.physics.uiowa.edu/~jbf/autoplot/javadoc2018/org/das2/util/filesystem/HtmlUtil.html#readToString">[view on old javadoc]</a> <a href="https://saturn.physics.uiowa.edu/svn/das2/dasCore/community/autoplot2011/trunk/dasCoreUtil/src/org/das2/util/filesystem/HtmlUtil.java">[view source]</a> <br> <br>