miércoles, 12 de septiembre de 2018

Mikrotik, queue tree basado en L7

En entradas anteriores compartí distintas formas de regular el tráfico a traves de queue simple y queue tree.
Ahora, experimentando con un nuevo router, el ccr 1009 (creedence) encontré este script para la versión 6 de ROS basado en Layer 7.

Como se puede observar, definir las reglas de layer 7 es a primera vista algo sencillo, sin embargo hay que comprobar luego que estas funcionen de acuerdo a nuestras necesidades.
Por el momento yo dejé el script tal como lo programó el autor y estoy monitoreando el tráfico observando la actividad en cada una de las ramas del queue tree.
Solo faltaría agregar una rama más que le dé baja prioridad al resto del tráfico desconocido.

El tráfico considerado es el siguiente:


VoIP
VPN
Tráfico de juegos (red xbox, red PS Wii, WoW, LoL, etc.)
Transmisión de video (netflix, youtube, etc.)
Navegación Http (conexiones de menos de 5 MB)
Protocolos de administración (ssh, telnet, etc.)
Servicios de administración (DNS, ICMP, etc.)
Todo lo que sobra se considera basura y lucha por lo que queda.



Agregamos dos clases nuevas de queue:
 /queue type
add kind=pfifo name=streaming-video-in pfifo-limit=500
add kind=pcq name=games-in-pcq pcq-classifier=dst-address \
    pcq-dst-address6-mask=64 pcq-rate=100k pcq-src-address6-mask=64 \
    pcq-total-limit=750000KiB
 creamos los filtros regex según nuestras necesidades:
/ip firewall layer7-protocol
add name=speedtest-servers regexp="^.*(get|GET).+speedtest.*\$"
add name=torrent-wwws regexp="^.*(get|GET).+(torrent|thepiratebay|isohunt|ente\
    rtane|demonoid|btjunkie|mininova|flixflux|vertor|h33t|zoozle|bitnova|bitso\
    up|meganova|fulldls|btbot|fenopy|gpirate|commonbits).*\$"
add name=torrent-dns regexp="^.+(torrent|thepiratebay|isohunt|entertane|demono\
    id|btjunkie|mininova|flixflux|vertor|h33t|zoozle|bitnova|bitsoup|meganova|\
    fulldls|btbot|fenopy|gpirate|commonbits).*\$"
add name=netflix regexp="^.*(get|GET).+(netflix).*\$"
add name=mp4 regexp="^.*(get|GET).+\\.mp4.*\$"
add name=swf regexp="^.*(get|GET).+\\.swf.*\$"
add name=flv regexp="^.*(get|GET).+\\.flv.*\$"
add name=video regexp="^.*(get|GET).+(\\.flv|\\.mp4|netflix|\\.swf).*\$"
Creamos las listas de direcciones de la salida a Internet y la red interna:
 /ip firewall address-list
add address=192.168.0.0/16 list=internal-nets
add address=138.121.228.0/24 list=external-nets
add address=192.168.254.90 comment="cliente 1" list=customer-servers
 creamos las reglas del firewall para marcar el tráfico conocido:
/ip firewall mangle
add action=mark-packet chain=prerouting comment=\
    "internal-traffic packet mark" dst-address-list=internal-nets \
    new-packet-mark=internal-traffic passthrough=no src-address-list=\
    internal-nets
add action=mark-packet chain=prerouting comment=\
    "customer-servers-out packet mark" new-packet-mark=customer-servers-out \
    passthrough=no src-address-list=customer-servers
add action=mark-packet chain=prerouting comment=\
    "customer-servers-in packet mark" dst-address-list=customer-servers \
    new-packet-mark=customer-servers-in passthrough=no
add action=mark-packet chain=prerouting comment="admin-in packet mark DNS" \
    in-interface=!ISP new-packet-mark=admin-in passthrough=no protocol=udp \
    src-port=53
add action=mark-packet chain=prerouting comment="admin-in packet mark snmp" \
    dst-port=161 in-interface=ISP new-packet-mark=admin-in passthrough=no \
    protocol=udp
add action=mark-connection chain=prerouting comment=\
    "Remote Protocols admin connection mark" new-connection-mark=admin port=\
    20,21,22,23,3389,8291 protocol=tcp
add action=mark-connection chain=prerouting comment=\
    "icmp connection mark as admin" new-connection-mark=admin protocol=icmp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="admin-in packet mark" \
    connection-mark=admin in-interface=ISP new-packet-mark=admin-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="admin-out packet mark" \
    connection-mark=admin new-packet-mark=admin-out passthrough=no
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark" dst-port=80 layer7-protocol=video \
    new-connection-mark=streaming-video protocol=tcp src-address-list=\
    internal-nets
add action=mark-packet chain=prerouting comment=\
    "streaming video in packet mark" connection-mark=streaming-video \
    in-interface=ISP new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=\
    "streaming video out packet mark" connection-mark=streaming-video \
    new-packet-mark=streaming-video-out passthrough=no
add action=mark-connection chain=prerouting comment=\
    "http traffic connection mark" dst-port=80,443 new-connection-mark=http \
    protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "http traffic connection mark" connection-bytes=5000000-4294967295 \
    dst-port=80,443 new-connection-mark=http-download protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="http in packet mark" \
    connection-mark=http in-interface=ISP new-packet-mark=http-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="http out packet mark" \
    connection-mark=http new-packet-mark=http-out passthrough=no
add action=mark-connection chain=prerouting comment=\
    "wow connetion mark as gaming" dst-port=\
    1119,3724,6112-6114,4000,6881-6999 new-connection-mark=games protocol=tcp \
    src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "eve online connetion mark as gaming" dst-address=87.237.38.200 \
    new-connection-mark=games src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "starcraft 2 connetion mark as gaming" dst-port=1119 new-connection-mark=\
    games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "heros of newerth connetion mark as gaming" dst-port=11031,11235-11335 \
    new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "steam connetion mark as gaming" dst-port=27014-27050 \
    new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "xbox live connetion mark as gaming" dst-port=3074 new-connection-mark=\
    games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "ps3 online connetion mark as gaming" dst-port=5223 new-connection-mark=\
    games protocol=tcp src-address-list=internal-nets
add action=mark-connection chain=prerouting comment=\
    "wii online connetion mark as gaming" dst-port=28910,29900,29901,29920 \
    new-connection-mark=games protocol=tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "games packet mark forever-saken-game" dst-address-list=external-nets \
    new-packet-mark=games-in passthrough=no src-address-list=\
    forever-saken-game
add action=mark-packet chain=prerouting comment="games packet mark wow" \
    dst-address-list=external-nets new-packet-mark=games-in passthrough=no \
    protocol=udp src-port=53,3724
add action=mark-packet chain=prerouting comment=\
    "games packet mark starcraft2" dst-address-list=external-nets \
    new-packet-mark=games-in passthrough=no protocol=udp src-port=1119,6113
add action=mark-packet chain=prerouting comment="games packet mark HoN" \
    dst-address-list=external-nets new-packet-mark=games-in passthrough=no \
    protocol=udp src-port=11031,11235-11335
add action=mark-packet chain=prerouting comment="games packet mark steam in" \
    dst-address-list=external-nets new-packet-mark=games-in passthrough=no \
    port=4380,28960,27000-27030 protocol=udp
add action=mark-packet chain=prerouting comment="games packet mark steam out" \
    dst-port=53,1500,3005,3101,3478,4379-4380,4380,28960,27000-27030,28960 \
    new-packet-mark=games-out passthrough=no protocol=udp src-address-list=\
    internal-nets
add action=mark-packet chain=prerouting comment="games packet mark xbox live" \
    dst-address-list=external-nets new-packet-mark=games-in passthrough=no \
    protocol=udp src-port=88,3074,3544,4500
add action=mark-packet chain=prerouting comment=\
    "games packet mark ps3 online" dst-address-list=external-nets \
    new-packet-mark=games-in passthrough=no protocol=udp src-port=\
    3478,3479,3658
add action=mark-packet chain=prerouting comment="games packet mark in" \
    connection-mark=games dst-address-list=external-nets new-packet-mark=\
    games-in passthrough=no
add action=mark-packet chain=prerouting comment="games packet mark out" \
    connection-mark=games new-packet-mark=games-out passthrough=no
add action=mark-packet chain=prerouting comment=\
    "voip-in packet mark teamspeak" dst-address-list=external-nets \
    new-packet-mark=voip-in passthrough=no protocol=udp src-port=9987
add action=mark-packet chain=prerouting comment=\
    "voip-out packet mark teamspeak" dst-port=9987 new-packet-mark=voip-out \
    passthrough=no protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "voip-out packet mark teamspeak" dst-address-list=external-nets \
    new-packet-mark=voip-in passthrough=no protocol=udp src-port=9987
add action=mark-packet chain=prerouting comment=\
    "voip-in packet mark ventrilo" dst-address-list=external-nets \
    new-packet-mark=voip-in passthrough=no protocol=udp src-port=3784
add action=mark-packet chain=prerouting comment=\
    "voip-out packet mark ventrilo" dst-port=3784 new-packet-mark=voip-out \
    passthrough=no protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "voip-in packet mark ventrilo" dst-address-list=external-nets \
    new-packet-mark=voip-in passthrough=no protocol=tcp src-port=3784
add action=mark-packet chain=prerouting comment=\
    "voip-out packet mark ventrilo" dst-port=3784 new-packet-mark=voip-out \
    passthrough=no protocol=tcp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="voip-in packet mark SIP" \
    dst-address-list=internal-nets new-packet-mark=voip-in passthrough=no \
    port=5060 protocol=tcp
add action=mark-packet chain=prerouting comment="voip-out packet mark SIP" \
    new-packet-mark=voip-out passthrough=no port=5060 protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="voip-in packet mark udp SIP" \
    dst-address-list=internal-nets new-packet-mark=voip-in passthrough=no \
    port=5004,5060 protocol=udp
add action=mark-packet chain=prerouting comment=\
    "voip-out packet mark udp SIP" new-packet-mark=voip-out passthrough=no \
    port=5004,5060 protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="voip-in packet mark RTP" \
    dst-address-list=internal-nets new-packet-mark=voip-in packet-size=\
    100-400 passthrough=no port=16348-32768 protocol=udp
add action=mark-packet chain=prerouting comment="voip-out packet mark RTP" \
    new-packet-mark=voip-in packet-size=100-400 passthrough=no port=\
    16348-32768 protocol=udp src-address-list=internal-nets
add action=mark-packet chain=prerouting comment="vpn-in packet mark GRE" \
    in-interface=ISP new-packet-mark=vpn-in passthrough=no protocol=gre
add action=mark-packet chain=prerouting comment="vpn-out packet mark GRE" \
    new-packet-mark=vpn-out passthrough=no protocol=gre
add action=mark-packet chain=prerouting comment="vpn-in packet mark ESP" \
    in-interface=ISP new-packet-mark=vpn-in passthrough=no protocol=ipsec-esp
add action=mark-packet chain=prerouting comment="vpn-out packet mark ESP" \
    new-packet-mark=vpn-out passthrough=no protocol=ipsec-esp
add action=mark-packet chain=prerouting comment=\
    "vpn-in packet mark VPN UDP ports" in-interface=ISP new-packet-mark=\
    vpn-in passthrough=no protocol=udp src-port=500,1701,4500
add action=mark-packet chain=prerouting comment=\
    "vpn-out packet mark VPN UDP ports" new-packet-mark=vpn-out passthrough=\
    no protocol=udp src-port=500,1701,4500
add action=mark-packet chain=prerouting comment="vpn-in packet mark PPTP" \
    in-interface=ISP new-packet-mark=vpn-in passthrough=no protocol=tcp \
    src-port=1723
add action=mark-packet chain=prerouting comment="vpn-out packet mark PPTP" \
    new-packet-mark=vpn-out passthrough=no protocol=tcp src-port=1723
add action=mark-packet chain=prerouting comment="all in" in-interface=ISP \
    new-packet-mark=in passthrough=no
add action=mark-packet chain=prerouting comment="all out" new-packet-mark=out \
    passthrough=no
 Finalmente creamos el queue tree:
/queue tree
add max-limit=10M name=in parent=global
add max-limit=10M name=out parent=global
add limit-at=3M max-limit=10M name=http-in packet-mark=http-in parent=in \
    priority=4 queue=default
add limit-at=4M max-limit=10M name=streaming-In packet-mark=\
    streaming-video-in parent=in priority=3 queue=streaming-video-in
add limit-at=500k max-limit=10M name=gaming-in packet-mark=games-in parent=in \
    priority=2 queue=games-in-pcq
add max-limit=10M name=download-in packet-mark=in parent=in queue=default
add max-limit=10M name=upload-out packet-mark=out parent=out queue=default
add limit-at=500k max-limit=10M name=gaming-out packet-mark=games-out parent=\
    out priority=2 queue=default
add limit-at=3M max-limit=10M name=http-out packet-mark=http-out parent=out \
    priority=4 queue=default
add limit-at=4M max-limit=10M name=streaming-Out packet-mark=\
    streaming-video-out parent=out priority=3 queue=default
add limit-at=1M max-limit=10M name=Clientes-In packet-mark=\
    customer-servers-in parent=in priority=1 queue=default
add limit-at=1M max-limit=10M name=Clientes-Out packet-mark=\
    customer-servers-out parent=out priority=1 queue=default
add limit-at=500k max-limit=10M name=voip-in packet-mark=voip-in parent=in \
    priority=1 queue=default
add limit-at=500k max-limit=10M name=vpn-in packet-mark=vpn-in parent=in \
    priority=2 queue=default
add limit-at=500k max-limit=10M name=voip-out packet-mark=voip-out parent=out \
    priority=1 queue=default
add limit-at=500k max-limit=10M name=vpn-out packet-mark=vpn-out parent=out \
    priority=2 queue=default
add limit-at=500k max-limit=10M name=admin-in packet-mark=admin-in parent=in \
    priority=1 queue=default
add limit-at=500k max-limit=10M name=admin-out packet-mark=admin-out parent=\
    out priority=1 queue=default

Aqui se puede observar un exceso en el trafico marcado como descargas:

 alarma en el tráfico http:

Tráfico total:

Fuente: http://gregsowell.com/?p=4665

viernes, 17 de noviembre de 2017

DHCP script para crear dhcpd.conf a partir de la db dhcpd.leases

Se me ha presentado el siguiente escenario: Tengo un servidor ubuntu que distribuye y rutea vlans y estoy en el medio de la elaboración de un plan de contingencia en el cual están comprendidos los router y los switch del MDF.
el BCP (Business Continuity Plan) esta pensado para una situación donde desaparece momentáneamente la topología lógica de Vlan. En el plan de continuidad permanece la subdivisión de redes pero esta ves por la asignación de los datos de configuración directamente por el isc-dhcp-server. Para ello el plan contempla dos tipos de incidentes:
1) fallas en el servidor distribuidor de vlans.
2) fallas en el switch principal con TRUNKs vlan.

 Para esto, configuré un servidor clon del distribuidor de vlans y me ví en la necesidad de crear un script que me permita obtener los datos de los hosts existentes conectados o apagados para asignarle estáticamente su pertenencia a la sub red.
De ello surge el script que explico a continuación.
En Ubuntu la base de datos del servidor dhcp esta en: /var/lib/dhcp/dhcpd.leases
y su configuración en:  /etc/dhcp/dhcpd.conf

el contenido de dhcpd.conf puede ser parecido a este:



y el contenido de dhcpd.leases:



Primero vamos a generar un archivo csv con el script explicado aqui:
http://jaskolowski.com.ar/2017/11/script-para-parsear-dhcpdlease.html

que es este:



nos genera un archivo con un contenido parecido a este:



ese archivo lo vamos a procesar con el siguiente script:



dejo el texto a continuación por si les resulta más fácil copy&paste:

#!/bin/bash
# script para crear dhcpd.conf a partir de la DB dhcpd.leases
#
# Creamos variables para obtener IP, MAC y Gateway de hosts
IP=(`cat lease.csv |cut -f1 -d";"`)
MAC=(`cat lease.csv |cut -f2 -d";"`)
GW=(`cat lease.csv |cut -f1 -d";"|sed 's:[^.]*$:1:'`)

# iniciamos el encabezado de dhcpd.conf

echo "ddns-update-style none;">dhcpd.conf
echo "Authoritative;">>dhcpd.conf
echo "ignore client-updates;">>dhcpd.conf


#creacion de un arreglo
for i in "${!IP[@]}"
do

# creamos cada lease por host
echo "# PC-${IP[$i]}">>dhcpd.conf
echo "Host PC-${IP[$i]} {">>dhcpd.conf
echo "        hardware ethernet               ${MAC[$i]};">>dhcpd.conf
echo "        fixed-address                   ${IP[$i]};">>dhcpd.conf
echo "        option subnet-mask              255.255.255.224;">>dhcpd.conf
echo "        option domain-name-servers      8.8.8.8,208.67.222.222;">>dhcp.conf
echo "        option routers                  ${GW[$i]};">>dhcpd.conf
echo "        default-lease-time -1;">>dhcpd.conf
echo "        max-lease-time -1;">>dhcpd.conf
echo "}">>dhcpd.conf
done
 Y eso nos da como resultado un archivo de configuración de dhcp server que va a asignar a cada host que tiene registrado por su mac la ip correspondiente a la sub red a la que pertenecía en un entorno vlan.

Adicionalmente se puede configurar una sub red para 30 ó 62 hosts que no hayan sido registrados en dhcpd.leases y aunque no esten en la sub red de su vlan, al menos tengan conexión hasta que se concluya el DRP.