Uploading meter reading data

Last week, I modified the MR Uploading module to include a check for the area's existence before the uploading proper started. In my blog entry, I prayed that I didn't introduce a bug "this time" but, guess what... (altogether now...) I DID!!! :-(

My latest quick fix failed to take Power Bills into consideration. Since Power Bills were read not by area but by reading day, the uploading process stops dead on its tracks at the area check.

I fixed it immediately.

Dvorak keyboard layout

There's been a sudden surge of the Dvorak keyboard layout in the plug mailing list lately, and it piqued my curiosity.

Now I switch to Dvorak when I can take time typing. Like now.

I'm still far from my qwerty speed though.

New thinkpad joins domain

I got my new company-issued Lenovo Thinkpad X61 7676A14 last Friday! Woot!

It's pre-loaded with "Vuisit-a" Vista Business, even though the sticker on the keyboard says Vista Basic. Nice, huh. Another pleasant surprise is that it's a Centrino Pro (I thought it was Centrino Duo only, not knowing of the existence of the Pro)! Centrino Pro is targeted at businesses. It's essentially the new Centrino Duo platform with an extra Intel Active Management chip. What this chip does is allow remote management and enhanced security features, even without an operating system.

Today I got the first chance to make it a domain member, but I kept getting "incorrect password" errors, despite disabling the usual "requiresign" etc... features. So I googled and found, among other links, this link.


- Click Start
- Click Control Panel
- Click System and Maintenance
- Click Administrative Tools
- Double-Click Local Security Policy
- In the left pane, click the triangle next to Local Policy
- In the left pane, click Security Options
- In the right pane near the bottom, double-click "Network security: LAN
manager authentication level"
- Click the drop-down box, and click "Send LM & NTLM - use NTLMv2 session
security if negotiated"
- Click OK
- Restart the computer


But since I was already in gpedit.msc (my usual approach), I looked for it in Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> Security Options -> Network Security: LAN Manager authentication level, and selected the same setting as above excerpt.

Voila! Instant access to the network! Next project is to replace our wireless access points with 802.11N... Woot!!!

BCMS: Bug Fix

My additional code for automatically Unvalidating zero and negative consumptions bit me.

I should've never touched the VALIDATED field if the computed consumption is negative (zero is now acceptable as valid), so I corrected it not to touch the VALIDATED field if the computed consumption is NOT negative. Otherwise, UNVALIDATE it.

I also corrected the Rover UPLOAD module to check for a matching area_id where searching for the account_no_id of the current consumer whose reading is being uploaded. I did this because some old account numbers in FB were equivalent to SF old account numbers when the hyphens are removed, and this screws up the upload process for some accounts.

I hope I didn't introduce new bugs this time.

BCMS: Data integrity fix

DPP and ELP approached me and reported that there was a NEGATIVE BILL last month. ELP was very quick to point out that i should put in code that checks for negative consumptions during bill computation.

My first reaction was unfavorable; I told him the flow of the system should be sufficient in preventing such problems. After some more thought, though, I realized that it's not a bad idea to do that as a "last line of defense" against this particular problem.

I then proceeded to search for bugs. During this search, I found out that the account is a mother meter and quickly realized that it was already PreVALIDATED by the program and retained its validation status even after the encoder has subtracted the sub-meters' consumptions. So I also added code that re-validates the reading after said input.

BCMS: Data Archiving note #1

select min(date_processed) from sfelapco.bill_detailed_summarized
11/1/2007
select min(date_processed) from old_data.bill_detailed_summarized
6/1/2006
select min(date_processed) from sfelapco2.bill_detailed_summarized
6/1/2006

select max(date_processed) from sfelapco.bill_detailed_summarized
1/20/2008
select max(date_processed) from old_data.bill_detailed_summarized
5/1/2007
select max(date_processed) from sfelapco2.bill_detailed_summarized
10/20/2007

select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-10-01' and '2007-10-31'
70187
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-09-01' and '2007-09-30'
70408
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-08-01' and '2007-08-31'
69943
select count(*) from sfelapco2.bill_detailed_summarized
where date_processed between '2007-07-01' and '2007-07-31'
69272

BCMS: Data Archiving

This entry serves as the main documentation for BCMS Data Archiving. Upon completion, it should have the following benefits:

1. Quicker backups.
2. Quicker data transfers/uploads.
3. Faster report generation.
4. Faster queries.

Now for the policies:

1. Following the method used for archiving the artran table, we will also archive these tables alongside artran_history to a new database called "history":
a. consumer_bill - Monthly (leave the previous month's data) - indexed on account_no_id, bill_no, bill_no_id
b. consumer_bill_details - Monthly (leave the previous month's data) - indexed on bill_no_id
c. bill_detailed_summarized - Monthly (leave the previous month's data) - indexed on account_no_id, bill_no
d. meter_reading - Monthly (leave the previous month's data)? - indexed on meter_reading_id, account_no_id, billing_month, actual_billing_month
e. payment - Monthly (leave the previous month's data) - indexed on account_no_id, ornumber, payment_date+teller_id
f. payment_details - Monthly (leave the previous month's data) - indexed on payment_detail_id, payment_id
g. payment_details_vat - Monthly (leave the previous month's data) - indexed on id, payment_detail_id
h. sql_logs, transaction_logs - Monthly (purge) - indexed on user_name, date?

2. All historical tables should be MyISAM.

3. Convert bill_detailed_summarized to MyISAM.

4. Update reports.

5. Immediately make a backup of the history database after archiving.

BCMS: Sacrificial Lamb

I heavily sacrified myself yesterday...

We were in a meeting to resolve the issues of LC's (load centers) servicing consumers in multiple areas. This problem surfaced because we wanted to be able to track consumers using feeders and load centers they are connected to, but our old feeder records are already several-years outdated (long ago, there were only 7 feeders but now there are 14 or 15), and this entails the need to update (and correct mistakes in) consumer records.

Several issues (some only related) came up:

1. RAJE (outsourced meter readers) will have to identify consumers which are recorded in the wrong areas (just a side problem).

2. Using new feeder-area groupings will be chaotic on RAJE's part (scheduling, etc.)

3. To achieve a proper hierarchical structure (Feeder - Area -LC - Meter), all individual LC's will have to be confined to one area each... an ENORMOUS task, to say the least, for the Engineering Department.

So I asked Eng'r. Lapuz if he needs to subdivide Feeders into areas, and when he said NO, I proposed that we:

1. Retain old feeder-area assignments as Revenue Areas, not Engineering Areas. The old FEEDER numbers will be renamed REVENUE GROUP numbers.

2. The FRANCHISE code at the beginning of each consumer's account number will be replaced with the FEEDER code/number they are serviced by/connected to.

The benefits of this scheme are:

1. NO CHANGES in RAJE's scheduling. They don't need to do any additional work!

2. NO TRANSFERRING of meters from LC to LC. They don't need to do any additional work!

3. NO DRASTIC CHANGES in Billing Reports and other stuff that refer to the old FEEDER-AREA numbers. (Note: Will assign modifications of report files to Rico.)

4. NO UNNECESSARY waste of paper (reports that fit 7/8 pages don't need to become 15 pages because of summarization by FEEDER).

The disadvantages:

1. SUBSTANTIALLY MORE WORK FOR moi!!! Aaaaaarrrgggghhh.... :

a. I need to modify the program to refer to old feeder numbers as revenue groups (I need to educate employees on referring to them from now on as such).

b. I need to restructure some tables in the database, which means I need to make sure the changes occur simultaneously at LH and RBM (2 San Fernando payment centers). Florida can be modified the day after.

c. For (b) to occur, I will have to create a temporary database for practice, along with a new set of code, just in case I commit a major blunder and have to revert to the old code.

Notes on 1.b.:
(i) Looks like uploading data from LH and RBM doesn't touch the consumer_master and consumer_meter_info tables, so I won't have to do a synchronized alteration of the table structures.

(ii) consumer_master: feeder_id and meter_information_id0 are both int(10) unsigned. Change these to transformer_id and pole_id, respectively. these have matching types. The feeder_id should then be picked up from the transformer_lup table. transformer here means LC (?)