Uploading meter reading data

Last week, I modified the MR Uploading module to include a check for the area's existence before the uploading proper started. In my blog entry, I prayed that I didn't introduce a bug "this time" but, guess what... (altogether now...) I DID!!! :-(

My latest quick fix failed to take Power Bills into consideration. Since Power Bills were read not by area but by reading day, the uploading process stops dead on its tracks at the area check.

I fixed it immediately.

Dvorak keyboard layout

There's been a sudden surge of the Dvorak keyboard layout in the plug mailing list lately, and it piqued my curiosity.

Now I switch to Dvorak when I can take time typing. Like now.

I'm still far from my qwerty speed though.

New thinkpad joins domain

I got my new company-issued Lenovo Thinkpad X61 7676A14 last Friday! Woot!

It's pre-loaded with "Vuisit-a" Vista Business, even though the sticker on the keyboard says Vista Basic. Nice, huh. Another pleasant surprise is that it's a Centrino Pro (I thought it was Centrino Duo only, not knowing of the existence of the Pro)! Centrino Pro is targeted at businesses. It's essentially the new Centrino Duo platform with an extra Intel Active Management chip. What this chip does is allow remote management and enhanced security features, even without an operating system.

Today I got the first chance to make it a domain member, but I kept getting "incorrect password" errors, despite disabling the usual "requiresign" etc... features. So I googled and found, among other links, this link.


- Click Start
- Click Control Panel
- Click System and Maintenance
- Click Administrative Tools
- Double-Click Local Security Policy
- In the left pane, click the triangle next to Local Policy
- In the left pane, click Security Options
- In the right pane near the bottom, double-click "Network security: LAN
manager authentication level"
- Click the drop-down box, and click "Send LM & NTLM - use NTLMv2 session
security if negotiated"
- Click OK
- Restart the computer


But since I was already in gpedit.msc (my usual approach), I looked for it in Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> Security Options -> Network Security: LAN Manager authentication level, and selected the same setting as above excerpt.

Voila! Instant access to the network! Next project is to replace our wireless access points with 802.11N... Woot!!!

BCMS: Bug Fix

My additional code for automatically Unvalidating zero and negative consumptions bit me.

I should've never touched the VALIDATED field if the computed consumption is negative (zero is now acceptable as valid), so I corrected it not to touch the VALIDATED field if the computed consumption is NOT negative. Otherwise, UNVALIDATE it.

I also corrected the Rover UPLOAD module to check for a matching area_id where searching for the account_no_id of the current consumer whose reading is being uploaded. I did this because some old account numbers in FB were equivalent to SF old account numbers when the hyphens are removed, and this screws up the upload process for some accounts.

I hope I didn't introduce new bugs this time.

BCMS: Data integrity fix

DPP and ELP approached me and reported that there was a NEGATIVE BILL last month. ELP was very quick to point out that i should put in code that checks for negative consumptions during bill computation.

My first reaction was unfavorable; I told him the flow of the system should be sufficient in preventing such problems. After some more thought, though, I realized that it's not a bad idea to do that as a "last line of defense" against this particular problem.

I then proceeded to search for bugs. During this search, I found out that the account is a mother meter and quickly realized that it was already PreVALIDATED by the program and retained its validation status even after the encoder has subtracted the sub-meters' consumptions. So I also added code that re-validates the reading after said input.

BCMS: Data Archiving note #1

select min(date_processed) from sfelapco.bill_detailed_summarized
11/1/2007
select min(date_processed) from old_data.bill_detailed_summarized
6/1/2006
select min(date_processed) from sfelapco2.bill_detailed_summarized
6/1/2006

select max(date_processed) from sfelapco.bill_detailed_summarized
1/20/2008
select max(date_processed) from old_data.bill_detailed_summarized
5/1/2007
select max(date_processed) from sfelapco2.bill_detailed_summarized
10/20/2007

select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-10-01' and '2007-10-31'
70187
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-09-01' and '2007-09-30'
70408
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-08-01' and '2007-08-31'
69943
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-07-01' and '2007-07-31'
69272

BCMS: Data Archiving

This entry serves as the main documentation for BCMS Data Archiving. Upon completion, it should have the following benefits:

1. Quicker backups.
2. Quicker data transfers/uploads.
3. Faster report generation.
4. Faster queries.

Now for the policies:

1. Following the method used for archiving the artran table, we will also archive these tables alongside artran_history to a new database called "history":
a. consumer_bill - Monthly (leave the previous month's data) - indexed on account_no_id, bill_no, bill_no_id
b. consumer_bill_details - Monthly (leave the previous month's data) - indexed on bill_no_id
c. bill_detailed_summarized - Monthly (leave the previous month's data) - indexed on account_no_id, bill_no
d. meter_reading - Monthly (leave the previous month's data)? - indexed on meter_reading_id, account_no_id, billing_month, actual_billing_month
e. payment - Monthly (leave the previous month's data) - indexed on account_no_id, ornumber, payment_date+teller_id
f. payment_details - Monthly (leave the previous month's data) - indexed on payment_detail_id, payment_id
g. payment_details_vat - Monthly (leave the previous month's data) - indexed on id, payment_detail_id
h. sql_logs, transaction_logs - Monthly (purge) - indexed on user_name, date?

2. All historical tables should be MyISAM.

3. Convert bill_detailed_summarized to MyISAM.

4. Update reports.

5. Immediately make a backup of the history database after archiving.