E2_LOADEPG - 26 March 2009  by ambrosa http://www.dreamboxonline.com

__author__ = "ambrosa http://www.dreamboxonline.com"
__version__ = "0.52 beta"
__copyright__ = "Copyright (C) 2008-2009 Alessandro Ambrosini"
__license__ = "CreativeCommons by-nc-sa http://creativecommons.org/licenses/by-nc-sa/3.0/"


 ** CHANGELOG **

26-March-2009 ver 0.52 beta
- updated skyitaliasat.bin to ver. 0.20. Now epg themes are downloaded and decoded (see dl_modules/sky_it.themes file)
- skyitalisat module now shows themes in epg descriptions
- correct a bug in ALL modules involved *-channel_list.conf -> default_provider (before the value was case sensitive, now it's case insensitive)


18-Jan-2009 ver. 0.51 beta
- If you have trouble with channel name and/or provider name, now in *-CHANNEL_LIST.CONF files you can use directly the channel SID
  Simply get the SID as is written in lamedb file and put it as a provider name with prefix 'sid='
  Example:
    # RAI4 , id=8
    8=2,rai 4,sid=2142:00820000:3070:013e:1:0
  Now it's assumed that RAI 4 channel has SID 2142:00820000:3070:013e:1:0
  The channel name will not be processed but it's use only for informational purpose

- In E2_LOADEPG.CONF added support for authorization/authentication with WebInterface

- Correct some bugs in UTF-8 and lamedb processing

- Correct some typos in log

- RAISAT download module (for RAI4 channel) is working again

- In BSKYBSAT.CONF removed USE_WEBIF option and added WAIT_FOR_MOTOR option

- In SKYITALIASAT.CONF removed USE_WEBIF option and added WAIT_FOR_MOTOR option

- Added in e2_coremod/ the original Python module SGMLLIB needed for simple HTML processing. This module is missing in Enigma2. It can be useful to create new download modules that need to process HTML web pages


06/01/2009 ver. 0.50 beta
Jump from 0.46 to 0.50 
Why ? Because I've rewritten e2_loadepg for handling correctly UTF-8 data.
You can think the version 0.50 as a new development branch focused on internationalization
This was mandatory for handling foreign characters outside ASCII charset

1) All configuration files (*.conf and *-channel_list.conf) now are expected to be encoded with UTF-8 charset (ASCII is fine: is an UTF-8 subset)
2) cache files (in cache/ directory) now are encoded in UTF-8 and no more in pure ASCII.
I've rewritten all download modules (skyitaliaweb, skyitaliasat, raisat, radiotimes ...) for handling the new charset: these modules now write cache files in UTF-8
3) e2_loadepg.log now is encoded in UTF-8 , log to video is in pure ASCII for safety reason (strange character >127 are replaced with '?')
4) EPG.DAT file is NOT in Unicode. It seems to be encoded in ISO-8859-1 or other monobyte charset. Now there is a new option in e2_loadepg.conf for
handling correct charset: EPG_CHARSET (default to 'iso-8859-1'). Now E2_LOADEPG read cache files (encoded in UTF-8) and re-encode data with 
EPG_CHARSET charset and write epg.dat file

5) module "skyitalia" (download SkyItalia EPG using web) is renamed to "skyitaliaweb"
6a) "skyitaliasat" module now recovers events only once. If event is recovered more than once, it's trashed. This reduce wrong events.
6b) "skyitaliasat.bin" updated to ver. 0.18 (minimized "Buffer Overflow" bug) and provided SH4 CPU version (see Examples/ dir. , thanks to Genge). Slower but safest.
7) module "bskyb" (download SKY-UK EPG using web) is renamed to "radiotimes"

8) added new download module "tvsajten" for Sweden/nordic channels (internet connection required)
9a) added new download module "bskybsat" + "bskybsat.bin rel. 0.11" for BSkyB (receive and decode OpenTV data from satellite transponder). Support for HDTV channels is ok (thanks to microch).
9b) provided "bskybsat.bin" SH4 CPU version (see Examples/ dir. , thanks to Genge)

10) in dl_modules/ added the program "sort-channel_list.py". It sort alphabetically *-channel_list.conf file . It can be useful for finding duplicated channel (channel with the same name) or if you like to have a channel list sorted alphabetically. Simply run it with ./sort-channel_list.py without any option : you get the help screen.

11) In Examples/ you can find the C source code of skyitaliasat.bin and bskybsat.bin to comply original Lukkino OpenTV code license (GPL)

I suppose there are bugs :-( , please let me know.


13/12/2008 ver. 0.46 beta
Solved a bug about title length > 250 characters
Added FLASHZA module in main distribution (Internet connection required)
Added RAISAT module for downloading Italy RAI4 epg (from RaiSat website: Internet connection required)


02/12/2008 ver. 0.45 beta
FLASHZA: added a new download module for South Africa. It get epg data from http://download.flash.za.org


19/11/2008 ver 0.45 beta
Correct a small bug in epg.dat creation (strip whitespace in provider name)
Added ENABLE_OSD option in e2_loadepg.conf : if Dreambox is switched on, see some info messages on the screen
Added ENABLE_OSD option in skyitaliasat.conf : if Dreambox is switched on, see some info messages on the screen
Removed STANDBY_AFTER_RESTART option in skyitaliasat.conf : now is always in "adaptative" mode
If there is a recording in place, now E2_LOADEPG does not a Restart GUI, you must does it manually after recording
If there is a recording in place, now Skyitaliasat module abort (it's impossible to change channel)
New skyitaliasat 0.16 module: now manage HDTV channel correctly


12/10/2009 ver. 0.44a beta
Correct a bug involved with Dreamboxedit 3.0 in lamedb file ("p:" is not at the start of the line)
Correct a bug in webif module


30/09/2009 ver. 0.44 beta
In e2_lodepg.conf aggiunta opzione LOG_LEVEL e nuova modalità per STANDBY_AFTER_RESTART (2 , adaptative)
In skyitaliasat.conf aggiunta nuova modalità per STANDBY_AFTER_DL (2 , adaptative) e aggiunte nuove opzioni ENABLE_CRCCHECK e ENABLE_RECOVERY
Aggiornato il programma skyitaliasat.bin alla rel. 0.15 : conteneva un baco nella routine di recovery.


31/08/2008 ver. 0.43 beta
Riscritto modulo SKYITALIASAT con incremento degli eventi scaricati
Aggiunte opzioni in skyitaliasa.conf
Possibilità di installare E2_LOADEPG anche in '/media/usb'


07/08/2008 ver. 0.42 beta
Aggiunto modulo "oztivo" (thanks to AJMILLER - Darwin/AU for suggestions and beta testing)
Modificato e2_coremod/stuff.py
Agggiunto esempio Example/e2_proxy.sh su come settare un eventuale HTTP proxy server


04/08/2008 ver. 0.41 beta
Aggiunto modulo "skyitaliasat"
Alcune modifiche cosmetiche al log
Modificati quasi tutti i .py
Modificato skyitalia.conf


27/07/2008 ver. 0.40 beta
Completamente riscritto il software usando la OOP (programmazione ad oggetti) e rendendolo modulare


13/07/2008 ver. 0.32 beta
Risolto bug nella gestione di excluded_sid


12/07/2008 ver. 0.31 beta
Modifica della licenza da GPL a "CreativeCommons by-nc-sa"


Aggiunto file di configurazione excluded_sid.conf e relativa opzione EXSID in e2_loadepg.conf. 
E2_LOADEPG assegna l'epg a tutti i canali aventi lo stesso nome. Questo file serve ad escludere gli eventuali SID a cui NON si vuole che venga assegnato l'epg. Il SID va ricavato manualmente consultando il file /etc/enigma2/lamedb


25/05/2008 ver. 0.30 beta
Introduzione del download dei dati epg via satellite grazie al programma accessorio 'e2_epgtrans' di MaxZ4.

Aggiunte nuove opzioni in e2_loadepg.conf :
EPGDATA_SOURCE
TRANSPONDER_CHANNEL
TRANSPONDER_BINARY_QUIET
DELETE_TRANSPONDER_DATA

Inoltre sono presenti due nuovi files:
 e2_epgtrans       il binario di MaxZ4 che scarica i dati epg da transponder
 e2_epgtrans.dic   il file testo che contiene il dizionario delle parole codificate
e2_epgtrans durante il suo funzionamento crea un file chiamato e2_epgtrans.dat che contiene i dati epg in formato SkyEPGIta
Se sono presenti parole codificate Huffmann sconosciute che andrebbero manualmente aggiunte in e2_epgtrans.dic, viene creato il file e2_epgtrans.err con gli errori. Questo file non viene cancellato per dare modo all'utente di investigare sui problemi.


24/05/2008 ver. 0.21 beta
Bug fix in channel_list_generator.py
La ricerca dei canali veniva effettuata dando per scontato che gli ID dei canali fossero nel range 1 ... 999 ma così non è.
Aggiunto in channel_list_generator.py la ricerca dei canali nel range 1 ... 99  e 5100 ... 5199


24/05/2008 ver. 0.20 beta
Da questa versione è OBBLIGATORIO avere attiva la WebInterface (WebIf) per poter usufruire di alcune funzioni come il Restart di Enigma, la messa in StandBy dopo il restart etc. etc.
Aggiunte le seguenti opzioni in e2_loadepg.conf :
USE_WEBIF
STANDBY_AFTER_RESTART


17/05/2008 ver. 0.11 beta
In channel_list.conf aggiunta opzione "3": come opzione "2" (forza lo scaricamento dell'XML) ma solo per la giornata corrente. Le altre giornate sono mantenute in cache come per opzione "1"
Gli XML sul sito SKY cambiano nel tempo e talvolta sono disallineati rispetto alla programmazione corrente.


11/05/2008 ver. 0.10 beta
Primo rilascio al pubblico



