Google
Edit File: queue.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>8.10. Queue — A synchronized queue class — Python 2.7.5 documentation</title> <link rel="stylesheet" href="../_static/default.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '../', VERSION: '2.7.5', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="../_static/jquery.js"></script> <script type="text/javascript" src="../_static/underscore.js"></script> <script type="text/javascript" src="../_static/doctools.js"></script> <script type="text/javascript" src="../_static/sidebar.js"></script> <link rel="search" type="application/opensearchdescription+xml" title="Search within Python 2.7.5 documentation" href="../_static/opensearch.xml"/> <link rel="author" title="About these documents" href="../about.html" /> <link rel="copyright" title="Copyright" href="../copyright.html" /> <link rel="top" title="Python 2.7.5 documentation" href="../index.html" /> <link rel="up" title="8. Data Types" href="datatypes.html" /> <link rel="next" title="8.11. weakref — Weak references" href="weakref.html" /> <link rel="prev" title="8.9. mutex — Mutual exclusion support" href="mutex.html" /> <link rel="shortcut icon" type="image/png" href="../_static/py.png" /> <script type="text/javascript" src="../_static/copybutton.js"></script> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="../py-modindex.html" title="Python Module Index" >modules</a> |</li> <li class="right" > <a href="weakref.html" title="8.11. weakref — Weak references" accesskey="N">next</a> |</li> <li class="right" > <a href="mutex.html" title="8.9. mutex — Mutual exclusion support" accesskey="P">previous</a> |</li> <li><img src="../_static/py.png" alt="" style="vertical-align: middle; margin-top: -1px"/></li> <li><a href="http://www.python.org/">Python</a> »</li> <li> <a href="../index.html">Python 2.7.5 documentation</a> » </li> <li><a href="index.html" >The Python Standard Library</a> »</li> <li><a href="datatypes.html" accesskey="U">8. Data Types</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="module-Queue"> <span id="queue-a-synchronized-queue-class"></span><h1>8.10. <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-mod docutils literal"><span class="pre">Queue</span></tt></a> — A synchronized queue class<a class="headerlink" href="#module-Queue" title="Permalink to this headline">¶</a></h1> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">The <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-mod docutils literal"><span class="pre">Queue</span></tt></a> module has been renamed to <tt class="xref py py-mod docutils literal"><span class="pre">queue</span></tt> in Python 3. The <a class="reference internal" href="../glossary.html#term-to3"><em class="xref std std-term">2to3</em></a> tool will automatically adapt imports when converting your sources to Python 3.</p> </div> <p><strong>Source code:</strong> <a class="reference external" href="http://hg.python.org/cpython/file/2.7/Lib/Queue.py">Lib/Queue.py</a></p> <hr class="docutils" /> <p>The <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-mod docutils literal"><span class="pre">Queue</span></tt></a> module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-class docutils literal"><span class="pre">Queue</span></tt></a> class in this module implements all the required locking semantics. It depends on the availability of thread support in Python; see the <a class="reference internal" href="threading.html#module-threading" title="threading: Higher-level threading interface."><tt class="xref py py-mod docutils literal"><span class="pre">threading</span></tt></a> module.</p> <p>The module implements three types of queue, which differ only in the order in which the entries are retrieved. In a FIFO queue, the first tasks added are the first retrieved. In a LIFO queue, the most recently added entry is the first retrieved (operating like a stack). With a priority queue, the entries are kept sorted (using the <a class="reference internal" href="heapq.html#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><tt class="xref py py-mod docutils literal"><span class="pre">heapq</span></tt></a> module) and the lowest valued entry is retrieved first.</p> <p>The <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-mod docutils literal"><span class="pre">Queue</span></tt></a> module defines the following classes and exceptions:</p> <dl class="class"> <dt id="Queue.Queue"> <em class="property">class </em><tt class="descclassname">Queue.</tt><tt class="descname">Queue</tt><big>(</big><em>maxsize=0</em><big>)</big><a class="headerlink" href="#Queue.Queue" title="Permalink to this definition">¶</a></dt> <dd><p>Constructor for a FIFO queue. <em>maxsize</em> is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If <em>maxsize</em> is less than or equal to zero, the queue size is infinite.</p> </dd></dl> <dl class="class"> <dt id="Queue.LifoQueue"> <em class="property">class </em><tt class="descclassname">Queue.</tt><tt class="descname">LifoQueue</tt><big>(</big><em>maxsize=0</em><big>)</big><a class="headerlink" href="#Queue.LifoQueue" title="Permalink to this definition">¶</a></dt> <dd><p>Constructor for a LIFO queue. <em>maxsize</em> is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If <em>maxsize</em> is less than or equal to zero, the queue size is infinite.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.6.</span></p> </dd></dl> <dl class="class"> <dt id="Queue.PriorityQueue"> <em class="property">class </em><tt class="descclassname">Queue.</tt><tt class="descname">PriorityQueue</tt><big>(</big><em>maxsize=0</em><big>)</big><a class="headerlink" href="#Queue.PriorityQueue" title="Permalink to this definition">¶</a></dt> <dd><p>Constructor for a priority queue. <em>maxsize</em> is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If <em>maxsize</em> is less than or equal to zero, the queue size is infinite.</p> <p>The lowest valued entries are retrieved first (the lowest valued entry is the one returned by <tt class="docutils literal"><span class="pre">sorted(list(entries))[0]</span></tt>). A typical pattern for entries is a tuple in the form: <tt class="docutils literal"><span class="pre">(priority_number,</span> <span class="pre">data)</span></tt>.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.6.</span></p> </dd></dl> <dl class="exception"> <dt id="Queue.Empty"> <em class="property">exception </em><tt class="descclassname">Queue.</tt><tt class="descname">Empty</tt><a class="headerlink" href="#Queue.Empty" title="Permalink to this definition">¶</a></dt> <dd><p>Exception raised when non-blocking <tt class="xref py py-meth docutils literal"><span class="pre">get()</span></tt> (or <tt class="xref py py-meth docutils literal"><span class="pre">get_nowait()</span></tt>) is called on a <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-class docutils literal"><span class="pre">Queue</span></tt></a> object which is empty.</p> </dd></dl> <dl class="exception"> <dt id="Queue.Full"> <em class="property">exception </em><tt class="descclassname">Queue.</tt><tt class="descname">Full</tt><a class="headerlink" href="#Queue.Full" title="Permalink to this definition">¶</a></dt> <dd><p>Exception raised when non-blocking <tt class="xref py py-meth docutils literal"><span class="pre">put()</span></tt> (or <tt class="xref py py-meth docutils literal"><span class="pre">put_nowait()</span></tt>) is called on a <a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-class docutils literal"><span class="pre">Queue</span></tt></a> object which is full.</p> </dd></dl> <div class="admonition-see-also admonition seealso"> <p class="first admonition-title">See also</p> <p class="last"><a class="reference internal" href="collections.html#collections.deque" title="collections.deque"><tt class="xref py py-class docutils literal"><span class="pre">collections.deque</span></tt></a> is an alternative implementation of unbounded queues with fast atomic <tt class="xref py py-func docutils literal"><span class="pre">append()</span></tt> and <tt class="xref py py-func docutils literal"><span class="pre">popleft()</span></tt> operations that do not require locking.</p> </div> <div class="section" id="queue-objects"> <span id="queueobjects"></span><h2>8.10.1. Queue Objects<a class="headerlink" href="#queue-objects" title="Permalink to this headline">¶</a></h2> <p>Queue objects (<a class="reference internal" href="#module-Queue" title="Queue: A synchronized queue class."><tt class="xref py py-class docutils literal"><span class="pre">Queue</span></tt></a>, <a class="reference internal" href="#Queue.LifoQueue" title="Queue.LifoQueue"><tt class="xref py py-class docutils literal"><span class="pre">LifoQueue</span></tt></a>, or <a class="reference internal" href="#Queue.PriorityQueue" title="Queue.PriorityQueue"><tt class="xref py py-class docutils literal"><span class="pre">PriorityQueue</span></tt></a>) provide the public methods described below.</p> <dl class="method"> <dt id="Queue.Queue.qsize"> <tt class="descclassname">Queue.</tt><tt class="descname">qsize</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.qsize" title="Permalink to this definition">¶</a></dt> <dd><p>Return the approximate size of the queue. Note, qsize() > 0 doesn’t guarantee that a subsequent get() will not block, nor will qsize() < maxsize guarantee that put() will not block.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.empty"> <tt class="descclassname">Queue.</tt><tt class="descname">empty</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.empty" title="Permalink to this definition">¶</a></dt> <dd><p>Return <tt class="docutils literal"><span class="pre">True</span></tt> if the queue is empty, <tt class="docutils literal"><span class="pre">False</span></tt> otherwise. If empty() returns <tt class="docutils literal"><span class="pre">True</span></tt> it doesn’t guarantee that a subsequent call to put() will not block. Similarly, if empty() returns <tt class="docutils literal"><span class="pre">False</span></tt> it doesn’t guarantee that a subsequent call to get() will not block.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.full"> <tt class="descclassname">Queue.</tt><tt class="descname">full</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.full" title="Permalink to this definition">¶</a></dt> <dd><p>Return <tt class="docutils literal"><span class="pre">True</span></tt> if the queue is full, <tt class="docutils literal"><span class="pre">False</span></tt> otherwise. If full() returns <tt class="docutils literal"><span class="pre">True</span></tt> it doesn’t guarantee that a subsequent call to get() will not block. Similarly, if full() returns <tt class="docutils literal"><span class="pre">False</span></tt> it doesn’t guarantee that a subsequent call to put() will not block.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.put"> <tt class="descclassname">Queue.</tt><tt class="descname">put</tt><big>(</big><em>item</em><span class="optional">[</span>, <em>block</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#Queue.Queue.put" title="Permalink to this definition">¶</a></dt> <dd><p>Put <em>item</em> into the queue. If optional args <em>block</em> is true and <em>timeout</em> is None (the default), block if necessary until a free slot is available. If <em>timeout</em> is a positive number, it blocks at most <em>timeout</em> seconds and raises the <a class="reference internal" href="#Queue.Full" title="Queue.Full"><tt class="xref py py-exc docutils literal"><span class="pre">Full</span></tt></a> exception if no free slot was available within that time. Otherwise (<em>block</em> is false), put an item on the queue if a free slot is immediately available, else raise the <a class="reference internal" href="#Queue.Full" title="Queue.Full"><tt class="xref py py-exc docutils literal"><span class="pre">Full</span></tt></a> exception (<em>timeout</em> is ignored in that case).</p> <p class="versionadded"> <span class="versionmodified">New in version 2.3: </span>The <em>timeout</em> parameter.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.put_nowait"> <tt class="descclassname">Queue.</tt><tt class="descname">put_nowait</tt><big>(</big><em>item</em><big>)</big><a class="headerlink" href="#Queue.Queue.put_nowait" title="Permalink to this definition">¶</a></dt> <dd><p>Equivalent to <tt class="docutils literal"><span class="pre">put(item,</span> <span class="pre">False)</span></tt>.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.get"> <tt class="descclassname">Queue.</tt><tt class="descname">get</tt><big>(</big><span class="optional">[</span><em>block</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#Queue.Queue.get" title="Permalink to this definition">¶</a></dt> <dd><p>Remove and return an item from the queue. If optional args <em>block</em> is true and <em>timeout</em> is None (the default), block if necessary until an item is available. If <em>timeout</em> is a positive number, it blocks at most <em>timeout</em> seconds and raises the <a class="reference internal" href="#Queue.Empty" title="Queue.Empty"><tt class="xref py py-exc docutils literal"><span class="pre">Empty</span></tt></a> exception if no item was available within that time. Otherwise (<em>block</em> is false), return an item if one is immediately available, else raise the <a class="reference internal" href="#Queue.Empty" title="Queue.Empty"><tt class="xref py py-exc docutils literal"><span class="pre">Empty</span></tt></a> exception (<em>timeout</em> is ignored in that case).</p> <p class="versionadded"> <span class="versionmodified">New in version 2.3: </span>The <em>timeout</em> parameter.</p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.get_nowait"> <tt class="descclassname">Queue.</tt><tt class="descname">get_nowait</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.get_nowait" title="Permalink to this definition">¶</a></dt> <dd><p>Equivalent to <tt class="docutils literal"><span class="pre">get(False)</span></tt>.</p> </dd></dl> <p>Two methods are offered to support tracking whether enqueued tasks have been fully processed by daemon consumer threads.</p> <dl class="method"> <dt id="Queue.Queue.task_done"> <tt class="descclassname">Queue.</tt><tt class="descname">task_done</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.task_done" title="Permalink to this definition">¶</a></dt> <dd><p>Indicate that a formerly enqueued task is complete. Used by queue consumer threads. For each <a class="reference internal" href="#Queue.Queue.get" title="Queue.Queue.get"><tt class="xref py py-meth docutils literal"><span class="pre">get()</span></tt></a> used to fetch a task, a subsequent call to <a class="reference internal" href="#Queue.Queue.task_done" title="Queue.Queue.task_done"><tt class="xref py py-meth docutils literal"><span class="pre">task_done()</span></tt></a> tells the queue that the processing on the task is complete.</p> <p>If a <a class="reference internal" href="#Queue.Queue.join" title="Queue.Queue.join"><tt class="xref py py-meth docutils literal"><span class="pre">join()</span></tt></a> is currently blocking, it will resume when all items have been processed (meaning that a <a class="reference internal" href="#Queue.Queue.task_done" title="Queue.Queue.task_done"><tt class="xref py py-meth docutils literal"><span class="pre">task_done()</span></tt></a> call was received for every item that had been <a class="reference internal" href="#Queue.Queue.put" title="Queue.Queue.put"><tt class="xref py py-meth docutils literal"><span class="pre">put()</span></tt></a> into the queue).</p> <p>Raises a <a class="reference internal" href="exceptions.html#exceptions.ValueError" title="exceptions.ValueError"><tt class="xref py py-exc docutils literal"><span class="pre">ValueError</span></tt></a> if called more times than there were items placed in the queue.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.5.</span></p> </dd></dl> <dl class="method"> <dt id="Queue.Queue.join"> <tt class="descclassname">Queue.</tt><tt class="descname">join</tt><big>(</big><big>)</big><a class="headerlink" href="#Queue.Queue.join" title="Permalink to this definition">¶</a></dt> <dd><p>Blocks until all items in the queue have been gotten and processed.</p> <p>The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls <a class="reference internal" href="#Queue.Queue.task_done" title="Queue.Queue.task_done"><tt class="xref py py-meth docutils literal"><span class="pre">task_done()</span></tt></a> to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, <a class="reference internal" href="#Queue.Queue.join" title="Queue.Queue.join"><tt class="xref py py-meth docutils literal"><span class="pre">join()</span></tt></a> unblocks.</p> <p class="versionadded"> <span class="versionmodified">New in version 2.5.</span></p> </dd></dl> <p>Example of how to wait for enqueued tasks to be completed:</p> <div class="highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">worker</span><span class="p">():</span> <span class="k">while</span> <span class="bp">True</span><span class="p">:</span> <span class="n">item</span> <span class="o">=</span> <span class="n">q</span><span class="o">.</span><span class="n">get</span><span class="p">()</span> <span class="n">do_work</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="n">q</span><span class="o">.</span><span class="n">task_done</span><span class="p">()</span> <span class="n">q</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_worker_threads</span><span class="p">):</span> <span class="n">t</span> <span class="o">=</span> <span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker</span><span class="p">)</span> <span class="n">t</span><span class="o">.</span><span class="n">daemon</span> <span class="o">=</span> <span class="bp">True</span> <span class="n">t</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">source</span><span class="p">():</span> <span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="n">q</span><span class="o">.</span><span class="n">join</span><span class="p">()</span> <span class="c"># block until all tasks are done</span> </pre></div> </div> </div> </div> </div> </div> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <h3><a href="../contents.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">8.10. <tt class="docutils literal"><span class="pre">Queue</span></tt> — A synchronized queue class</a><ul> <li><a class="reference internal" href="#queue-objects">8.10.1. Queue Objects</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="mutex.html" title="previous chapter">8.9. <tt class="docutils literal"><span class="pre">mutex</span></tt> — Mutual exclusion support</a></p> <h4>Next topic</h4> <p class="topless"><a href="weakref.html" title="next chapter">8.11. <tt class="docutils literal"><span class="pre">weakref</span></tt> — Weak references</a></p> <h3>This Page</h3> <ul class="this-page-menu"> <li><a href="../bugs.html">Report a Bug</a></li> <li><a href="../_sources/library/queue.txt" rel="nofollow">Show Source</a></li> </ul> <div id="searchbox" style="display: none"> <h3>Quick search</h3> <form class="search" action="../search.html" method="get"> <input type="text" name="q" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="../genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="../py-modindex.html" title="Python Module Index" >modules</a> |</li> <li class="right" > <a href="weakref.html" title="8.11. weakref — Weak references" >next</a> |</li> <li class="right" > <a href="mutex.html" title="8.9. mutex — Mutual exclusion support" >previous</a> |</li> <li><img src="../_static/py.png" alt="" style="vertical-align: middle; margin-top: -1px"/></li> <li><a href="http://www.python.org/">Python</a> »</li> <li> <a href="../index.html">Python 2.7.5 documentation</a> » </li> <li><a href="index.html" >The Python Standard Library</a> »</li> <li><a href="datatypes.html" >8. Data Types</a> »</li> </ul> </div> <div class="footer"> © <a href="../copyright.html">Copyright</a> 1990-2019, Python Software Foundation. <br /> The Python Software Foundation is a non-profit corporation. <a href="http://www.python.org/psf/donations/">Please donate.</a> <br /> Last updated on Jul 03, 2019. <a href="../bugs.html">Found a bug</a>? <br /> Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> </body> </html>