Organisieren eines regulären Log Digests mit Python und Ansible am Beispiel eines Sternchens

Als ich dieses Tool erstellte, war ich mit Logwatch nicht vertraut. Ich wollte die Situation mit den Protokollen auf meinen Servern im Allgemeinen sehen und habe dieses Fahrrad gemacht. Ich denke, dass dieser Mechanismus Anfängern helfen kann, die alternativen Möglichkeiten von Ansible zu verstehen.

Gebrauchte Softwareprodukte:

• Python 2.7.14
• ansible 2.3
• Sternchen-Server basierend auf FreePBX 13

Der Mechanismus besteht aus zwei Teilen - einem Python-Skript, das die Protokolldatei verarbeitet und einen Bericht an die E-Mail sendet, und einem Playbook zum Sammeln von Protokollen von Servern und zum Übertragen dieser Protokolle zur Verarbeitung an das Skript.

Playbook selbst:

--- - name: parseastlogs hosts: production_asterisks vars: date: "{{ lookup('pipe', 'date +%Y%m%d') }}" ipaddr: "{{ ansible_default_ipv4.address }}" tasks: - debug: var=date - debug: var=ipaddr - fetch: src: /var/log/asterisk/full-{{ date }} dest: /tmp/full-{{ date }}-{{ ipaddr }} flat: yes - local_action: "shell /etc/ansible/localscripts/astReporter.py {{ ipaddr }} full-{{ date }}-{{ ipaddr }}"    ,            fetch.        -   flat,       . 

Skript:

 #!/usr/bin/python2.7 import re from collections import Counter import yagmail import sys import datetime import os servername=sys.argv[1] yag=yagmail.SMTP(user='ansible@2.1',password=None,host='192.168.2.1',port='25',smtp_starttls=False,smtp_set_debuglevel=0,smtp_skip_login=True) recipients=['user@mydomain.local'] filename=sys.argv[2] workdir='/tmp/' log_file=open(workdir+filename,'r')#sys.argv[1] loglist=list() verbosity=['NOTICE','ERROR','WARNING'] regexes = ["Call from '.*' .* to extension '.*' rejected because extension not found in context '.*'.", "Identifier \d+, identifier_type \d+ not found in identifier list", "Invalid result identifier \d+ passed in aMYSQL_clear", "This function can only be used on SIP channels.", "fwrite() returned error: Broken pipe", "CDR requires a value \(CDR\(variable\)=value\)", "Received SIP subscribe for peer without mailbox: .*", "Removed interface '.*' from queue '.*'", "Peer '.*' is trying to register, but not configured as host=dynamic", "Registration from '.*' failed for '.*' - Peer is not supposed to register", "Unable to join queue '.*'", "Attempt to pause interface Local/@from-queue/n, not found", "PRESENCE_STATE unknown", "EXTENSION_STATE requires an extension", "Prodding channel '.*' failed", "Channel '.*' sent to invalid extension but no invalid handler: context,exten,priority=.*", "Can't send 10 type frames with PJSIP", "Attempt to pause interface .+, not found", "Attempt to unpause interface .+, not found", "no samples for ulawtolin", "Could not find matching INVITE transaction for CANCEL request", "Peer '.*' is now Reachable. \(.*\)", "Peer '.*' is now UNREACHABLE! Last qualify: .*", "Registration from .* failed for '.*' - Wrong password", "Retransmission timeout reached on transmission .*", "no samples for alawtolin", "Peer '.*' is now Lagged. \(\d+ms / \d+ms\)", "Call completed to .*", "Invalid retrytime at line \d+ of .*", "Not accepting call completion offers from call-forward recipient .*", "No such context '.*' for macro '.*'\. Was called by .*", "[pP]ickup .* attempt by .*", "Call failed to go through, reason \(5\) Remote end is Busy", "Deprecated syntax found\..* ", "No digits dialed for atxfer.", "Unable to create channel of type 'SIP' \(cause \d+ - Subscriber absent\)", "'tls' is not a valid transport type when tlsenable=no\. If no other is specified, the defaults from general will be used\.", "'tcp' is not a valid transport type when tcpenable=no\. If no other is specified, the defaults from general will be used\.", "Queued call to .* expired without completion after \d+ attempts", "Re-invite to non-existing call leg on other UA. SIP dialog .*\. Giving up.", "Channel .* not found! Variable 'BLKVM' not set to .*\.", "Remote host can't match request CANCEL to call .*\. Giving up\.", "Unable to execute query \[.*\]", "SQL Exec Direct failed \(-1\)!\[", "SQL Execute returned an error .*", " -- Re-registration for .*", "Outbound Registration\: Expiry for .* is \d+ sec \(Scheduling reregistration in \d+ s\)", "Correct auth, but based on stale nonce received from '.*'", "Unable to write frametype: 2", "Received response: \"Forbidden\" from '\".*\" .*'", "Huh\? Not an RDNIS SIP header .*", "Hanging up call .* - no reply to our critical packet .*", "Cancelling retransmit of OPTIONs \(call id .*\) ", "Still have a QUALIFY dialog active, deleting", "The use of '_\.' for an extension is strongly discouraged and can have unexpected behavior. Please use '_X\.' instead at line .*", "Context '.*' tries to include nonexistent context '(.*)'", "aMYSQL_query: mysql_query failed\. Error: Duplicate entry .*", "RTCP SR transmission error to .*, rtcp halted Operation not permitted", "Failed to write frame to '.*': Resource temporarily unavailable", "Unable to forward frametype: 2", "Timeout on .* on non-critical invite transaction.", "Unexpected control subclass '\d+'", "Context '.*' for macro '.*' lacks .*", "No response received from '.*' on registration attempt to '.*', retrying in '\d+'", "Unknown RTP codec 90 received from '.*'", "Invalid extension '.*', but no rule 'i' or 'e' in context '.*'", "Added interface '.*' to queue '.*'", "Exceptionally long voice queue length queuing to .*", "Request from '.*' failed for '.*' \(callid: .*\) - No matching endpoint found", "Junk at the beginning of frame \d+", "Unable to register extension at line .*", "Unable to register extension '.*' priority .* in '.*', already in use", "Unable to load config file .*", 'Purely numeric hostname \(\d+\), and not a peer--rejecting!', "Unknown directive .* at line \d+ .*", "Context '.*' already included in '.*' context on include at line \d+ of .*", "No closing parenthesis found\? '.*' at line \d+ of .*", "Can't use '.*' priority on the first entry at line \d+ of .*", "The .* options are deprecated. Please see UPGRADE.txt for information", "Can't use '.*' priority on the first entry at line \d+ of .*", "Call failed to go through, reason .*", "Unable to open .*", "Playback of message .* failed", "File .* does not exist in any format", "Playback failed on .* for .*", "Adding .* to .*", "Can't move channel. One or both is dead .*", "Unable to complete call pickup of .*", "Pickup .* failed by .*", "No entry in voicemail config file for .*", " \-\- Registration for '.*' timed out, trying again \(Attempt #[0-9]+\)", "Disconnecting call .* for lack of RTP activity in [0-9]+ seconds", "Failed to initialize .* declining image stream", "Can't send 10 type frames with SIP write", "Set requires an '=' to be a valid assignment.", "Timeout but no rule 't' or 'e' in context .*", "RTCP RR transmission error to .*, rtcp halted Operation not permitted", "Unable to acquire target extension for attended transfer.", "Unterminated comment detected beginning on line"] notmatched=list() matchedregex=list() combinedre="("+")|(".join(regexes) + ")" #           .        . class logitem: ldatetime=None ltype=None lchan=None lsource=None lcontent=None def __init__(self,ldate,ltype,lch,lsrc,lcont): self.ldatetime=ldate self.ltype=ltype self.lchan=lch self.lsource=lsrc self.lcontent=lcont #        def main(): i=0 dumpchanstart=False for line in log_file: #   Dumpchan if 'app_dumpchan.c' in line: dumpchanstart = 0 for line in log_file: if dumpchanstart==2: break if "================================================================================" in line: dumpchanstart+=1 #    ,    [ if line[0]=="[": buf=re.split('] ', line) #     verbosity if any(x in buf[1] for x in verbosity): buf1spl=buf[1].split('-') channum='' buf2spl=re.split('c: ',buf[2]) if len(buf1spl)>1: channum=buf1spl[1] itemcheck=logitem(buf[0][1:],buf[1].split('[')[0],channum,buf2spl[0],buf2spl[1]) filtration(itemcheck.lcontent) #      matchedregex,        notmatched,  . def filtration(logitem): found = False for regexitem in regexes: if re.match(regexitem,logitem): found= True matchedregex.append(regexitem) continue if found==False: notmatched.append(logitem) #    .   if..elif      def genContents(): content=u"<html>" content+=u"<head></head>" content+=u"<body>" content+=u"<h1>Log digest for " +servername+ u"</h1>" content+=u"<table style='width: 60%;' border = \"1\" cellpadding = \"1\"'>" for key, val in Counter(matchedregex).most_common(): if u"rejected because extension not found in context" in key: key=u"<b style='background-color:#e8edff'>"+key+u"</b>" elif u"No matching endpoint found" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" elif u"No closing parenthesis " in key: key = u"<b style='background-color:#98862a'>" + key + u"</b>" elif u"Wrong password" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" elif u"Unterminated comment detected beginning on line" in key: key=u"<b style='background-color:#fd6161'>"+key+u"</b>" content += u"<tr>" content+= u"<td>"+key+u"</td><td>" +str(val) +u"</td>" content += u"</tr>" content+=u"</table><br>" content+=u"<h1>not matched logitems:</h1><br>" for item in notmatched: content+=item content+=u"</body>\n" content+=u"</html>\n" return content main() cont=genContents() yag.send(recipients,subject='Log digest for '+servername,contents=cont) os.remove(workdiir+filename) 

Das Skript wird mit dem folgenden Befehl gestartet: / usr / local / bin / ansible-playbook /etc/ansible/playbooks/parseastlogs.yml

Das Ergebnis des Befehls ist eine Anzahl von Buchstaben (ein Buchstabe für jeden Server aus der Gruppe Production_asterisk) mit ungefähr dem folgenden Inhalt:



Wenn Sie Fragen oder Anregungen haben, bin ich bereit, diese zu beantworten.