Organiser un résumé de journal régulier avec python et ansible en utilisant un astérisque comme exemple

Lorsque j'ai créé cet outil, je ne connaissais pas la surveillance des journaux. Je voulais voir la situation avec les journaux sur mes serveurs en général, et j'ai donc fait ce vélo. Je pense que ce mécanisme peut aider les débutants à comprendre les possibilités alternatives d'ansible.

Produits logiciels utilisés:

• python 2.7.14
• ansible 2.3
• serveurs d'astérisque basés sur FreePBX 13

Le mécanisme se compose de deux parties: un script python qui traite le fichier journal et envoie un rapport au courrier électronique, et un playbook pour collecter les journaux des serveurs et les transférer vers le script pour traitement.

Playbook lui-même:

--- - name: parseastlogs hosts: production_asterisks vars: date: "{{ lookup('pipe', 'date +%Y%m%d') }}" ipaddr: "{{ ansible_default_ipv4.address }}" tasks: - debug: var=date - debug: var=ipaddr - fetch: src: /var/log/asterisk/full-{{ date }} dest: /tmp/full-{{ date }}-{{ ipaddr }} flat: yes - local_action: "shell /etc/ansible/localscripts/astReporter.py {{ ipaddr }} full-{{ date }}-{{ ipaddr }}"    ,            fetch.        -   flat,       . 

Script:

 #!/usr/bin/python2.7 import re from collections import Counter import yagmail import sys import datetime import os servername=sys.argv[1] yag=yagmail.SMTP(user='ansible@2.1',password=None,host='192.168.2.1',port='25',smtp_starttls=False,smtp_set_debuglevel=0,smtp_skip_login=True) recipients=['user@mydomain.local'] filename=sys.argv[2] workdir='/tmp/' log_file=open(workdir+filename,'r')#sys.argv[1] loglist=list() verbosity=['NOTICE','ERROR','WARNING'] regexes = ["Call from '.*' .* to extension '.*' rejected because extension not found in context '.*'.", "Identifier \d+, identifier_type \d+ not found in identifier list", "Invalid result identifier \d+ passed in aMYSQL_clear", "This function can only be used on SIP channels.", "fwrite() returned error: Broken pipe", "CDR requires a value \(CDR\(variable\)=value\)", "Received SIP subscribe for peer without mailbox: .*", "Removed interface '.*' from queue '.*'", "Peer '.*' is trying to register, but not configured as host=dynamic", "Registration from '.*' failed for '.*' - Peer is not supposed to register", "Unable to join queue '.*'", "Attempt to pause interface Local/@from-queue/n, not found", "PRESENCE_STATE unknown", "EXTENSION_STATE requires an extension", "Prodding channel '.*' failed", "Channel '.*' sent to invalid extension but no invalid handler: context,exten,priority=.*", "Can't send 10 type frames with PJSIP", "Attempt to pause interface .+, not found", "Attempt to unpause interface .+, not found", "no samples for ulawtolin", "Could not find matching INVITE transaction for CANCEL request", "Peer '.*' is now Reachable. \(.*\)", "Peer '.*' is now UNREACHABLE! Last qualify: .*", "Registration from .* failed for '.*' - Wrong password", "Retransmission timeout reached on transmission .*", "no samples for alawtolin", "Peer '.*' is now Lagged. \(\d+ms / \d+ms\)", "Call completed to .*", "Invalid retrytime at line \d+ of .*", "Not accepting call completion offers from call-forward recipient .*", "No such context '.*' for macro '.*'\. Was called by .*", "[pP]ickup .* attempt by .*", "Call failed to go through, reason \(5\) Remote end is Busy", "Deprecated syntax found\..* ", "No digits dialed for atxfer.", "Unable to create channel of type 'SIP' \(cause \d+ - Subscriber absent\)", "'tls' is not a valid transport type when tlsenable=no\. If no other is specified, the defaults from general will be used\.", "'tcp' is not a valid transport type when tcpenable=no\. If no other is specified, the defaults from general will be used\.", "Queued call to .* expired without completion after \d+ attempts", "Re-invite to non-existing call leg on other UA. SIP dialog .*\. Giving up.", "Channel .* not found! Variable 'BLKVM' not set to .*\.", "Remote host can't match request CANCEL to call .*\. Giving up\.", "Unable to execute query \[.*\]", "SQL Exec Direct failed \(-1\)!\[", "SQL Execute returned an error .*", " -- Re-registration for .*", "Outbound Registration\: Expiry for .* is \d+ sec \(Scheduling reregistration in \d+ s\)", "Correct auth, but based on stale nonce received from '.*'", "Unable to write frametype: 2", "Received response: \"Forbidden\" from '\".*\" .*'", "Huh\? Not an RDNIS SIP header .*", "Hanging up call .* - no reply to our critical packet .*", "Cancelling retransmit of OPTIONs \(call id .*\) ", "Still have a QUALIFY dialog active, deleting", "The use of '_\.' for an extension is strongly discouraged and can have unexpected behavior. Please use '_X\.' instead at line .*", "Context '.*' tries to include nonexistent context '(.*)'", "aMYSQL_query: mysql_query failed\. Error: Duplicate entry .*", "RTCP SR transmission error to .*, rtcp halted Operation not permitted", "Failed to write frame to '.*': Resource temporarily unavailable", "Unable to forward frametype: 2", "Timeout on .* on non-critical invite transaction.", "Unexpected control subclass '\d+'", "Context '.*' for macro '.*' lacks .*", "No response received from '.*' on registration attempt to '.*', retrying in '\d+'", "Unknown RTP codec 90 received from '.*'", "Invalid extension '.*', but no rule 'i' or 'e' in context '.*'", "Added interface '.*' to queue '.*'", "Exceptionally long voice queue length queuing to .*", "Request from '.*' failed for '.*' \(callid: .*\) - No matching endpoint found", "Junk at the beginning of frame \d+", "Unable to register extension at line .*", "Unable to register extension '.*' priority .* in '.*', already in use", "Unable to load config file .*", 'Purely numeric hostname \(\d+\), and not a peer--rejecting!', "Unknown directive .* at line \d+ .*", "Context '.*' already included in '.*' context on include at line \d+ of .*", "No closing parenthesis found\? '.*' at line \d+ of .*", "Can't use '.*' priority on the first entry at line \d+ of .*", "The .* options are deprecated. Please see UPGRADE.txt for information", "Can't use '.*' priority on the first entry at line \d+ of .*", "Call failed to go through, reason .*", "Unable to open .*", "Playback of message .* failed", "File .* does not exist in any format", "Playback failed on .* for .*", "Adding .* to .*", "Can't move channel. One or both is dead .*", "Unable to complete call pickup of .*", "Pickup .* failed by .*", "No entry in voicemail config file for .*", " \-\- Registration for '.*' timed out, trying again \(Attempt #[0-9]+\)", "Disconnecting call .* for lack of RTP activity in [0-9]+ seconds", "Failed to initialize .* declining image stream", "Can't send 10 type frames with SIP write", "Set requires an '=' to be a valid assignment.", "Timeout but no rule 't' or 'e' in context .*", "RTCP RR transmission error to .*, rtcp halted Operation not permitted", "Unable to acquire target extension for attended transfer.", "Unterminated comment detected beginning on line"] notmatched=list() matchedregex=list() combinedre="("+")|(".join(regexes) + ")" #           .        . class logitem: ldatetime=None ltype=None lchan=None lsource=None lcontent=None def __init__(self,ldate,ltype,lch,lsrc,lcont): self.ldatetime=ldate self.ltype=ltype self.lchan=lch self.lsource=lsrc self.lcontent=lcont #        def main(): i=0 dumpchanstart=False for line in log_file: #   Dumpchan if 'app_dumpchan.c' in line: dumpchanstart = 0 for line in log_file: if dumpchanstart==2: break if "================================================================================" in line: dumpchanstart+=1 #    ,    [ if line[0]=="[": buf=re.split('] ', line) #     verbosity if any(x in buf[1] for x in verbosity): buf1spl=buf[1].split('-') channum='' buf2spl=re.split('c: ',buf[2]) if len(buf1spl)>1: channum=buf1spl[1] itemcheck=logitem(buf[0][1:],buf[1].split('[')[0],channum,buf2spl[0],buf2spl[1]) filtration(itemcheck.lcontent) #      matchedregex,        notmatched,  . def filtration(logitem): found = False for regexitem in regexes: if re.match(regexitem,logitem): found= True matchedregex.append(regexitem) continue if found==False: notmatched.append(logitem) #    .   if..elif      def genContents(): content=u"<html>" content+=u"<head></head>" content+=u"<body>" content+=u"<h1>Log digest for " +servername+ u"</h1>" content+=u"<table style='width: 60%;' border = \"1\" cellpadding = \"1\"'>" for key, val in Counter(matchedregex).most_common(): if u"rejected because extension not found in context" in key: key=u"<b style='background-color:#e8edff'>"+key+u"</b>" elif u"No matching endpoint found" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" elif u"No closing parenthesis " in key: key = u"<b style='background-color:#98862a'>" + key + u"</b>" elif u"Wrong password" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" elif u"Unterminated comment detected beginning on line" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" content += u"<tr>" content+= u"<td>"+key+u"</td><td>" +str(val) +u"</td>" content += u"</tr>" content+=u"</table><br>" content+=u"<h1>not matched logitems:</h1><br>" for item in notmatched: content+=item content+=u"</body>\n" content+=u"</html>\n" return content main() cont=genContents() yag.send(recipients,subject='Log digest for '+servername,contents=cont) os.remove(workdiir+filename) 

le script est lancé par la commande: / usr / local / bin / ansible-playbook /etc/ansible/playbooks/parseastlogs.yml

Le résultat de la commande sera un nombre de lettres (une lettre pour chaque serveur du groupe production_asterisk) avec approximativement le contenu suivant:



Si vous avez des questions ou des suggestions, je suis prêt à y répondre.