python多线程-queue队列类型优先级队列，FIFO,LIFO

灰太狼 2022-06-15 09:40 286阅读 0赞

Queue在python3中重命名为queue，在python2到python3转换中可以自动转换

队列可应用在多个生产者多个消费者的模型中[<<详情参考>>][Link 1]，并且在多线程中可用于线程之间数据信息的安全交换通信，防止冲突。  
在队列中已经实现多线程的锁机制

#### 队列Queue提供三种队列类型 ####

主要区别是操作顺序的不同：  
1.class Queue.Queue(maxsize=0)  
FIFO队列，先进先出，maxsize定义队列长度，若maxsize等于0或者小于0，则该队列长度为无限，默认是无限  
2.class Queue.LifoQueue(maxsize=0)  
LIFO 后进先出，类似于堆栈，若maxsize等于0或者小于0，则该队列长度为无限，默认是无限  
3.class Queue.PriorityQueue(maxsize=0)  
优先级队列，值越低则优先级越高，若maxsize等于0或者小于0，则该队列长度为无限，默认是无限

#### 异常： ####

1.exception Queue.Empty  
当get()或get\_nowait() 触发  
2.exception Queue.Full  
当put()或put\_nowait()触发

#### 三种队列均提供如下方法： ####

1.Queue.qsize() 队列长度，注：当qsize长度大于0时，get()并不一定成功，当长度小于maxsize时，put（）也不一定成功  
2.Queue.empty() 是否队列为空，空True,反之False  
3.Queue.full() 队列是否满，满True,反之False  
4.Queue.get(\[block\[, timeout\]\])  
获取队列一个排队对象，timeout等待时间，如果get等待timeout时间，仍为空，则抛出异常。默认block为true timeout为null，一直等待有对象返回  
5.Queue.get\_nowait() 相当Queue.get(False)  
6.Queue.put(item)  
将一个对象写入队列，timeout等待时间，如果一个put等待了timeout时间，仍为满，则会抛出异常。默认block为true timeout为null，一直等待对象写入  
7.Queue.put\_nowait(item) 相当Queue.put(item, False)  
8.Queue.task\_done()  
9.Queue.join() 相当与threading中将进程挂起，知道将子线程中的所有任务都处理完成后，才退出join，进行后面的步骤

例如模型：

def worker():  
while True:  
item = q.get()  
do\_work(item)  
q.task\_done()

q = Queue()  
for i in range(num\_worker\_threads):  
t = Thread(target=worker)  
t.daemon = True  
t.start()

for item in source():  
q.put(item)

q.join()

#### 1.FIFO队列先进先出： ####

#coding=utf8
    import Queue
    
    queuelist = Queue.Queue()
    
    for i in range(5):
        if not queuelist.full():
           queuelist.put(i)
           print "put list : %s ,now queue size is %s "%(i,queuelist.qsize())
    
    while not queuelist.empty():
        print "get list : %s , now queue size is %s"%(queuelist.get(),queuelist.qsize())

输出：

put list : 0 ,now queue size is 1 
    put list : 1 ,now queue size is 2 
    put list : 2 ,now queue size is 3 
    put list : 3 ,now queue size is 4 
    put list : 4 ,now queue size is 5 
    get list : 0 , now queue size is 4
    get list : 1 , now queue size is 3
    get list : 2 , now queue size is 2
    get list : 3 , now queue size is 1
    get list : 4 , now queue size is 0

#### 2.LIFO队列先进后出： ####

#coding=utf8
    import Queue
    
    queuelist = Queue.LifoQueue()
    
    for i in range(5):
        if not queuelist.full():
           queuelist.put(i)
           print "put list : %s ,now queue size is %s "%(i,queuelist.qsize())
    
    while not queuelist.empty():
        print "get list : %s , now queue size is %s"%(queuelist.get(),queuelist.qsize())

输出：

put list : 0 ,now queue size is 1 
    put list : 1 ,now queue size is 2 
    put list : 2 ,now queue size is 3 
    put list : 3 ,now queue size is 4 
    put list : 4 ,now queue size is 5 
    get list : 4 , now queue size is 4
    get list : 3 , now queue size is 3
    get list : 2 , now queue size is 2
    get list : 1 , now queue size is 1
    get list : 0 , now queue size is 0

3.优先队列PriorityQueue  
ps:这个优先队列有些垃圾，没有直接传入优先级的参数  
示例：

#coding=utf8
    import Queue
    import random
    
    queuelist = Queue.PriorityQueue()
    
    for i in range(5):
        if not queuelist.full():
            x=random.randint(1,20)
            y=random.randint(1,20)
            print y,x
            queuelist.put({y:"th-%d"%x})
    
    while not queuelist.empty():
        print "get list : %s , now queue size is %s"%(queuelist.get(),queuelist.qsize())

输出：

11 5
    16 16
    6 13
    5 19
    12 20
    get list : {
       5: 'th-19'} , now queue size is 4
    get list : {
       6: 'th-13'} , now queue size is 3
    get list : {
       11: 'th-5'} , now queue size is 2
    get list : {
       12: 'th-20'} , now queue size is 1
    get list : {
       16: 'th-16'} , now queue size is 0

优先级优先值传入的几种传入方法：

将queuelist.put(\{y:”th-%d”%x\})可替换为如下：

1.queuelist.put((y,”th-%d”%x))  
2.queuelist.put(\[y,”th-%d”%x\])

若不可替换queuelist.put(\{y:”th-%d”%x\})修改为如下put，没有想要的优先队列效果：

1.queuelist.put(y,”th-%d”%x)  
2.queuelist.put(\{y,”th-%d”%x\})

#### 检票综合示例： ####

模拟检票过程  
排3队进行检票，一个队列一个线程  
票有编号  
检票时间有固定窗口，若时间到了通知队列关闭

#coding=utf8
    
    import Queue
    import threading
    import time
    
    exitsingle = 0
    
    class myThread(threading.Thread):
        def __init__(self, threadname, queuelist):
            threading.Thread.__init__(self)
            self.threadname = threadname
            self.queuelist = queuelist
    
        def run(self):
            print "Starting queue %s"%self.threadname
            queue_enter(self.threadname, self.queuelist)
            time.sleep(1)
            print "close " + self.threadname
    
    def queue_enter(threadname, queuelist):
        while not exitsingle:
            queueLock.acquire()
            if not workQueue.empty():
                data = queuelist.get()
                queueLock.release()
                print "%s check ticket %s" % (threadname, data)
            else:
                queueLock.release()
            time.sleep(1)
    
    
    threadList = ["list-1", "list-2", "list-3"]
    queueLock = threading.Lock()
    workQueue = Queue.Queue()
    threads = []
    
    queueLock.acquire()
    for num in range(100001,100020):
        workQueue.put(num)
    
    queueLock.release()
    
    print "start .."
    
    for name in threadList:
        thread = myThread( name, workQueue)
        thread.start()
        threads.append(thread)
    
    while not workQueue.empty():
         pass
    
    exitsingle = 1
    
    for t in threads:
        t.join()
    print "stop enter.."

输出：

start ..
    Starting queue list-1
    list-1 check ticket 100001
    Starting queue list-2
    list-2 check ticket 100002
    Starting queue list-3
    list-3 check ticket 100003
    list-1 check ticket 100004
    list-2 check ticket 100005
    list-3 check ticket 100006
    list-1 check ticket 100007
    list-2 check ticket 100008
    list-3 check ticket 100009
    list-1 check ticket 100010
    list-2 check ticket 100011
    list-3 check ticket 100012
    list-1 check ticket 100013
    list-2 check ticket 100014
    list-3 check ticket 100015
    list-1 check ticket 100016
    list-2 check ticket 100017
    list-3 check ticket 100018
    list-1 check ticket 100019
    close  list-3
    close  list-1
    close  list-2
    stop enter..

[Link 1]: http://blog.csdn.net/comprel/article/details/72811436