(The following is an edited version of the transcript of the Senate Banking Committee hearing on June 21, 2017, looking into the so-called hacking of BPI. Certain Tagalog words were translated into English whenever possible and as appropriate.):
BPI President Cezar P. Consing explains the internal data processing error on June 6, 2017
The data processing error caused mis-postings in the account of about 1.5 million of our eight million clients. These mispostings were in amounts much, much smaller than some of the figures that were circulating prominently in social and other media.
To fix the problem, we had to take down our electronic channels—services related to our ATM cards, internet banking, mobile banking, for a total of 26 hours spread, over a period of 37 hours.
This inconvenienced an even greater number of clients, many of whom are your (the senators’) constituents. As a bank that has prided itself in bringing convenience to clients over and above basic banking services, we deeply regret the inconvenience caused by this time gap.
Human error, not a hack
We have a robust system that handles an average of 3.5 million transaction per day. Our investigation revealed that ours was a case of human error and not a hacking. There was no breach of data privacy.
We shared with the regulators the actions we took to make things right, from the correction of the mis-postings to the reversals or waivers of penalties and charges to the longer banking hours, to the continuous stream of client advisories.
BPI’s Ramon Jocson elaborates on what happened
The bank’s processes are fully automated or nearly all are automated. We do batch processing. Every night we have an end of day processing. At 8 pm we process all transactions. By 10 pm are our central data base do the updates. The central server automatically updates all the balances.
And then what we do is we take a copy, a flash copy of that database at that point in time and then we open all electronic channels. It’s fast. In 15 minutes, we do the downloading and then the switching.
This is a closed system. It is not connected to anything external. It is not connected to the internet. Its only connection is our mainframe computers, each running on servers and these have their own network.
All other transactions we have are just passed on as batch files. There is no direct connection to the internet. That is what we mean by a closed system.
On June 6, we had to reconcile a report. What happens is we have depositors who go abroad and then they use the ATM machines abroad to withdraw money. We needed to reconcile a report from May 26 to May 29 (from) one of our correspondent banks.
That was the request given by our shared operations group, this was then given to one of our lead technical persons for the system. There are only 12 people who are trained in this system. Out of the 12, only two have access to the system. The specialist in question is one of the two. So she was assigned to extract a report and she was instructed to take it from the backup files.
To get it from the backup files, she had to route a request and inform her superior that she’s going to do something in the backup environment. What she did was, because maybe of expediency, since she already has access to the production file and since she knows that it can be done more expediently in the production file, the temptation was there and in order to rush the report, she said, “I can do it in production.” So she generated the report in production and extracted the file. Now the file contained instead of May 26 to May 29, she entered in the online system the dates of April 27 to May 2.
In effect, na-extract lahat ng transactions of April 27 to May 2 and it created a file in the “IST”. That’s the name of the system. The file was automatically named using the systems naming convention. It gave the date, June 6, 2017. So, in effect, this was submitted at around 3 p.m. and it concluded at around 6 p.m., and she left the file there.
At around 8 p.m., the batch extraction took place, but since there was already a file created, it created another file called “point one.”
In Windows, that would be called Copy One.
We are using Unix, here the convention is point one. So it doesn’t replace the file but instead just renames it to another file. It is given an extension called .1 (point one).
At around 10 p.m., the extracted April 27 to May 2 file was used by the system, instead of the June 6 (file). So in effect, the balances were updated using transactions from April 27 to May 2, instead of June 6.
This covered only transactions in our own ATMs, our point of sale systems, and our cash acceptance machines. You can only withdraw money from ATMs. You deposit money through our cash acceptance machines.
An illustrative example: If a client’s balance was padded, it could happen if you had P10,000. However, if you withdrew cash of 5,000, 3,500, and 4,000, respectively on April 27, 28, and 29, you would suddenly notice that your balance would be P2,500 because these transactions were re-posted.
No crossing of transactions to other clients
The transactions re-posted were your own transactions but they were at a certain point in time in the past. Never did transactions from other clients cross to other clients. So, these were all your transactions in the past. They were simply re-posted.
Now for a credit example, if you deposited in our cash acceptance machine, say P100,000 on April 29, and another P6,000 on April 29, suddenly you will see that your balance would be P116,000.
The average debit, when we looked at it, out of the 1.5 million clients which Mr. Consing mentioned, was around an average debit of P7,700; average credit was P7,200.
Impossible to post billions
There were people saying that they had P12 billion, right? For you to reach P1 billion for the five days, you would have had to deposit P200 million per day. Now, with our cash acceptance machine, the highest bill you can feed is P1,000. You feed it every two seconds. So, theoretically, in one minute, you can feed around P30,000 times 60 minutes. In one hour, P1.8 million. In order for you to do P200 million a day, you would have to do it across five machines, non-stop for 24 hours, feeding P1,000 (bills). That is quite impossible. Our cassettes get filled at P4 million (maximum). So those people claiming they posted P12 billion, P8 billion, P1 billion – they are all false. The documents that they presented to the public and to the TV stations, radio stations were all doctored and faked.
As Mr. Consing mentioned, we did recovery over a span of 37 hours. There were lapses during that time, so the downtime in our channels was 24 hours. What we did was to re-post back.
Avoiding occurrence
What will we do to prevent a recurrence? During that day, we had in our ATM system transactions that surged from an average of P2 billion to P10 billion, because of the five days. Instead of one day it became five days.
So, what we have imposed now, is an automatic circuit breaker, whenever transactions reach P10 billion or more than one million transactions. It will disable the production system and ask, is this enough or not?
The second thing that we’re doing is addressing our mean time to recover. Why did it take so long to restore the system. Because, we had to post back five days worth of transactions and one day of valid transaction. So, six days worth of transactions ginawa namin in one day kaya natagalan.
We’re gonna put in more restore points. Ang importante with any complex system is to have the ability to recover. In the future, we will avoid this recurrence but this also gives us something to think about that maybe what we need to do is to put in more restore points. So, at least every step maka-recover kami kaagad.
Now, the third thing that we need to do, and are doing already, is go 24/7. We are upgrading our deposit systems to go 24/7. We intend to optimize the memo posting capability as we upgrade.
So, by the end of this year, our deposit system will be 24/7. Meaning to say, we do not have to put down the online systems while we are running batch runs. We are now optimizing the system.
Why didn’t BPI catch it early?
The last thing that we need to do, and probably this is the question that you will ask, bakit hindi namin nahuli ng mas maaga? In fact, the warning flags came from our clients asking questions about their balances. And we learned about this at around, maybe 6:30.
There is a process that we do, which is called reconciliation. Before the branches open at around 8:00 a.m., our shared services group takes a look at cash on hand kasi kailangan mag-deliver kami ng cash sa different branches. Kailangan magbalanse ang ledger namin. And then kailangan we deliver cash to the branches that need cash. We fill up ATMs to those ATMs that need cash. We could have seen that in the report, kasi makikita na 10 billion ang withdrawal, hindi balance. So, what we’re gonna do, instead of 8:00 a.m. we’re gonna move that up to 4 a.m.
The moment it’s free, we stop our batch run, we do the check first before we open up our channels.
Because of an error in judgment of one of our programmers where the programmer in question thought that maybe this is a faster way of doing it in delivering the assignment that I was tasked to do. Because of that there was an unscheduled file generation which then caused us to post wrong entries which we were able to then reverse. And these are the remedial actions that we’re doing to prevent a recurrence.
Joseph Albert Gotuaco explains how BPI coped with the situation
In BPI, we serve about eight million customers. The most basic of the services are what you call deposit, withdrawal and the lending transaction.
And those services are provided in the most basic of channels which we call our branches. At BPI, we call it the branch of account where you can do all the basic services. We then extended these services to add convenience to our customers. And these convenience services are added transactions such as fund transfers, remittance, payments of bills.
We also have what we call an inter-branch transaction. At BPI, we call this “Bank Anywhere,” which means to say, if you open your account at one branch, you should be able to transact at another branch where you did not open that account for free, for no extra charge. So, that is added convenience to our customers. In addition to that, for our customers who do not wish to bank in the branch, we also offer services that they can do at home, where you can bank online. When you are stuck in traffic, you can bank using our mobile platforms. We also have ATMs which is the most used electronic channel, and POS machines for your credit and debit cards.
BPI branches process 850,000 transactions per day
Now, let me describe the branch. All our branches look the same but it’s important to make some distinctions here.
First of all, we have 827 branches in BPI across our parent bank and our subsidiaries. Those branches are generally open between 9 a.m. and 4:30 p.m., seven to seven and a half hours a day. That may change. If it’s in a mall, it might slide but generally, seven to seven and a half hours a day.
And the systems that we have in the branch are such that we can process transactions at a speed of about 850,000 transactions per day and this is our primary channel for all our banking services.
If this is the bank also where you opened your account, where a BPI customer opened an account, this is where all your primary balances are kept. It is a procedure in this branch that every day, before they open, certain printouts are made. While we open at 9, employees typically come in at 7:30 and 8.
It’s interesting to note, one of the things that they print are the account balances of all their customers so that if on any given day, there’s an error of the balance, they can always revert to printouts.
These are hard copies of customer data from the prior day. And that’s why the branch of account is critical in an emergency situation because if it goes offline, there’s certain data that’s protected.
And this branch network is particularly important, because on the days of June 7th and June 8th when we knew that the electronic channels were down, this was the system on which we felt we could rely on to service our customers for all their banking needs.
Channels affected
Let me describe to you the alternative channels that were affected on June 7 and June 8. So the first one was the automatic teller machine—ATM network. This is the most used convenience channel that we have outside of the branches. BPI has about 2,300 of these machines with 4.6 million users and it processes about 400,000 transactions a day, approximately.
The next channel is our cash acceptance machines. It’s the exact opposite of an ATM. Instead of dispensing cash, it takes in cash, particularly for clients who do not wish to come into the branch or want to bank off-hours. It is a relatively newer machine that’s been in service since only 2011 or so. We only have 800 of them, about only one million users and we process basically 5% of transactions on CAMS as we do on ATMs.
BPI Express Online
The next channel is our online. It’s branded “BPI Express Online.” This is used on a computer or a laptop. We have 1.2 million active users doing 40,000 transactions per day.
And finally, we have our mobile application system. It’s called “express mobile,” 1.1 million users, 70,000 transactions a day.
So if I could put this into perspective, when you combine the transactional history of these four channels that were affected by the June 7 and June 8 incident, you’re looking at approximately half a million or between 500,000 to 600,000 transactions per day that were impaired during the event.
When we described the branch banking system, our branches have a capacity to do 850,000 transactions. So knowing that our electronic channels were down, we immediately mobilized to extend banking hours to allow our customers to come to their branch banking network to fulfill their financial transactions.
When the event also broke out, when we found out about it on June 7th, it was very important for us to understand from talking to Mr. Jocson’s team what was going on because we were very urgently in a position, very pressed to provide to our customers also an idea of what was going on—give them comfort on what the root cause was and give them an outlook.
What was also very important was to communicate,,, to our branch network because it was them who would had to remedy the situation and deal with customers. Particularly important here, was knowledge of the fact of the mispostings because remember, that was what would allow them to use their backup data to service their customers for their banking needs.
We boosted our customer care systems. We don’t just serve customers through our branches on June 7 and June 8, we also have our telephonic and our social media customer service infrastructure and they were boosted during the events of June 7 and 8 and thereafter. And obviously, for our customers affected by this and unable to make payments, we reversed fees, particularly in relation to BPI-related services when they were impacted by delayed payments.
We extended banking hours. Our branches typically close at 4:30. So we created three additional hours of banking time of processing capacity in our branches on Wednesday, June 7; Thursday, June 8; and Friday, June 9.
And in addition to that, just for extra measure, 200 of our 800 branches, or 237 to be precise, were opened on Saturday for additional banking for a full day.
What this does in terms of providing relief to our customers. If you take the combined impact of extending to 7:30 for three days and the opening of the Saturday, what we’ve effectively done is create the capacity for 1.7 million monetary transactions.
That is well in excess of the 540,000 or the 600,000 transactions that was foregone on the 26 hours or so that we lost electronic channels. And so, while this is a rather simple calculation. it does illustrate the thinking behind the team when we mobilized to open branches and create capacity for our customers.
(May I illustrate) the intensity of our communications during the five-day period—June 7 through June 13.
We issued a total 17 public advisories. It was pretty rapid-fire during June 7th. In one instance, one announcement following another in 45 minutes but we sustained it for five days.
And once we were in recovery mode, the relationship with our customers, we were trying to make a little bit more personal and we began issuing letters and contacting them one by one.
And obviously, the channels we used were our online social media and any avenues we had through print and broadcast.
You can see the intensity of the communications was as high as on June 7th and June 8th and it died down as we went into normalcy on Friday, June 9.
And finally, we mobilized our call center personnel. We have 400 of these personnel. We boosted by 25 headcount during the peak hours of the incident to make sure that we could field our calls and ensure there was no backlogs.
Committee Chairman Escudero wants guarantee what happened was not hacking
Escudero: Can you say 100% definite, it was not a hack.
Ramon Jocson. 100% definite it was not a hack. This was reported to me at 7 a.m. and by 7:30 a.m., I concluded that this was not a hack.
BPI has a cyber security operation center. That cyber security operation center tracks 20,000 events per second: withdraw ka sa ATM, log on ka sa online, may tellering system, those are events.
Those 20,000 events per second. We basically use tools to look for aberrant behavior. Meaning to say, mayroon bang nag-escalate ng privileges, may pumapasok ba?
During that time, noong nakita ko na ganito ang nangyari, ibig sabihin niyan mayroon nang leakage palabas. Ibig sabihin siguro kung na-hack kami, mayroon traffic palabas. So tiningnan ko kaagad, iyong traffic. Kasi we have that in our CSOC reports traffic for the last three days especially for the last two hours, there was none.
The second thing that I did, was to call up our service providers. We have two service providers, FireEye Mandiant and IBM.
FireEye Mandiant does threat analytics for us. Meaning to say, aside from monitoring the system, they also look out for messages at the dark web, anything, any noise if you were hacked. So negative on that area.
IBM manages our security operation center for us. I also asked them for any other reports outside of those that were given me and there were none. So by that time, I concluded that given that this was a closed system, walang connection sa Internet, the only possible thing na pwedeng pumasok was that somebody was in control of a work station and pounding out commands but I saw from the network wala namang pumapasok.
Then I concluded, that it was not a hack. Plus, this was verified by our service providers, IBM and FireEye.
THE CHAIRMAN (SEN. ESCUDERO). You mentioned a circuit breaker to avoid future occurrences, what would be the practical effect of that, may delay ba sa posting? Ano ang epekto noon from the point of view of the depositor?
Jocson: Ibig sabihin lang niyan sa aming automated processing, instead of straight-through, mayroon kaming gap na dadagdag na 10 minutes. So, let’s say, kapag nakita na, let’s say, the transaction volume was extraordinary, mag-stop iyong automated production at that time and then it will ask the operator, yes or no to proceed. So, that basically means that he needs to look at the reason why the volume ballooned to that size. If everything is okay, all that person needs to do is say yes then it will proceed, probably, 10 minutes delay. If abnormal, then, we go to—to take a look at what the root cause was.
Escudero: Marami ng mga kumpanyang gumagamit ng ATM para sa payroll nila, what was the situation with respect to payrolls that needed to be made during the time or period affected? Has everything been resolved by now?
JOSEPH ALBERT Gotuaco: Yes, we have automated payment system, that were affected by the downtime particularly on Friday as we were recovering—as you could imagine one to 26 hours loss of processing time. But ultimately, the payrolls were affected. We had some delays but most of our customers or a great majority of our customers were finally settled in terms of payroll by the time the week ended.
Escudero: Of the 540 transactions that you saw may have been affected, not necessarily all, because if they did not make a transaction during that period, walang galawan sa account nila?
Gotuaco: Wala po. Yes, that’s correct.
Escudero: So of the 540 transactions, lahat na ba ng claim na-settle na ng BPI as of today?
Gotuaco: In fact, it is difficult to know what the totality of claims is at this point because we are still early since the incident. But all claims that are brought to our knowledge, we have engaged customers. We, in fact, have invested quite a bit of time in understanding how they were affected by bounced checks, by delayed payments of interests and principal, by premium payments on insurance contracts that may have been due and where we have identified those situations. In fact, we have reversed fees already such many of our customers may not even have known they were impacted by the incident.
Escudero: As a process, kung may complaint ang isang depositor na nalaman niya next week, next month na maaaring naapektuhan sila, ano iyong prosesong pagdadaanan niya para gumawa ng claim sa BPI?
Gotuaco: Marami ho tayong proseso. In fact, even during the incident, I wish to emphasize that our branches were always open. So that would have been your first line of defense, so to speak, if you are a customer. The branches were open and they were open for extended hours and they were open on Saturday.
If that’s just the first option, there are other ways that we have reached out to customers. Many of our customers have branch managers and relationships managers that know them. We initiated call-outs to our customers once normalcy occurred to reach out to make sure that they are comfortable that they haven’t been affected.
Escudero: Just to clarify. There is no prescriptive period for this. Meaning, if you discover it a year from now, they can still go to you and point it out and prove their claim, if at all?
Gotuaco: What we do is I wish to assure you that every single customer case, not just with respect to the June 7 and June 8 incident, is important to us. And BPI will make every effort to consider the merits of each case to make sure that our customers are comfortable and satisfied.
Escudero: What if the payment or the premium that needed to be paid is with respect to a third party corporation and as a result of which, fees or penalties or interest was imposed on the depositor? Meaning, it’s not a BPI-affiliated company.
Gotuaco: Yes, po.
Escudero: How do you go through the process of “reversing” whatever such penalties or interest rates?
Gotuaco: That’s correct. It is very easy for us to understand, the fees as it relates to our own services. I think you are asking how does it relate to other third parties.
And what we have done for all our customers, we have reached out to them and if they do have a case of that third party merchant, we would like to engage them on the facts and circumstances of their inconvenience and we will—I assure you and our customers that we will do our best to make sure that they are taken care of.
Escudero: Finally, with respect to the specialist that committed the error whether through negligence or because, as you said, Mon, she was in a rush, what action has been taken, if any, insofar as she is concerned by the bank?
Jocson: The investigations have been pointed that this particular person is to blame. This particular person has also owned up to committing the mistake. In terms of the investigation, it’s still ongoing because, what we’re looking at now are certain processes and protocols that we need to strengthen so that this will not happen again. But as of the moment until this is concluded, the said person, has been reassigned to another area and all access rights to our systems have been taken out.
Sen. franklin Drilon: It is clear that it was an error in judgment of a programmer?
Jocson: Yes. And if I may answer the question in a different way. I was a programmer once, I was young once and sometimes—you make mistakes.
And this particular person, has been three years with us. She topped her programming class. So there is always the zeal to be able to do things faster. So I attribute this, to a lapse in judgment.
Not malicious, nothing to gain at all.
Drilon: Yes. Now, there were some talks about unauthorized access to your clients deposits, no such thing happened here?
Jocson: No such thing. I can categorically say, that all the clients saw with their own postings or their own transactions which were done a month ago, but other than that, they would not be able to see other information outside of theirs.
As far as the bank is concerned, the data that was affected, this transaction file is anonymized. It only contains transaction codes, transaction amounts. Not names. No money was lost.
Drilon: So none of your clients lost money?
Jocson: Yes. No money was lost.
Drilon: Well, you have done corrective measures and I assume that you can assure the public that—I don’t know if you can assure the public that such innocent errors in judgment in the future will not happen. Whatever was created by human beings is subject to frailties and, in this particular case, I don’t know what other assurances you can give the public having done everything that you have described.
Jocson: These are complex systems that we are working with, a combination of process, people and technology. You need trusted people to be able to further the process. What’s important to a bank like us and to an organization like us is the ability to recover, the ability to recover, the ability to preserve the integrity of the data at any time such that we can go back and then recover.
So I think that from a risk management focus, that’s what we are doing. You cannot avoid having complex systems, because banking uses advanced technologies. But my assurance to the public, is, we are investing to ensure that our mean time to recover will be faster. But there will be no doubt that we will be able to recover should any other similar incident occur in the future.
Escudero: I’d like to distinguish and have a dichotomy in our discussions with respect to BPI and BDO, that’s why if you will bear with me, sir, we’ll just finish up with them, BPI, before proceeding with the incidents relating to BDO.
May I address my questions to Assistant deputy Governor Fonacier?
Ma’am, the BSP as the regulator of banks in general, BPI more specifically, did its own investigation on the matter. And may I get your input insofar as the perspective of the BSP is regard to this incident, as the regulators of BPI.
CHUCHI FONACIER. Good morning.
Actually, on the side of the BSP, we also conducted our own investigation but we haven’t really completed yet the full investigation. But we can say at this point is that, there was really no hacking nor a computer glitch. It seems really a human error, but the concern of the BSP in this case—the situation of BPI in that particular situation is, the internal controls in that—why the internal control failed to detect right away the mistake or the errors, so that’s basically it.
Escudero: Insofar as liabilities under existing regulation is concerned, what, if any, are there?
Fonecier: We haven’t completed, as I mentioned, the investigation but it would seem from the initial investigation that we’ve done is that, in the case of BPI, this situation is, there was no loss to any client. But our emphasis is that, client should be guided by BPI on how to go about—if they have some concerns on their accounts. So I think basically, that’s what the bank should do – be very clear to clients in addressing complaints related to this particular situation.
Escudero: Without preempting the results of your final investigation, how was thus far BPI been able to perform insofar as rectifying the incident. In 37 hours, Mr. Consing, is that correct?
CEZAR Consing: Yes, 37 hours from start to finish.
Escudero: What’s the benchmark with respect to this or similar situations, iyong rectification procedure, good batting average? Kumbaga sa good, better, best, is that good?
Fonecier: I think, it’s too difficult to make a judgment at this point because we haven’t really completed yet the investigation. So maybe we’ll leave it at that but the call really is for BPI to be very clear about handling the effects of such a situation, especially to the clients.
Escudero: Now, quite frankly, Assistant Governor Fonacier, at 540,000 transactions involving approximately 1.7 million—how many depositors?
Consing: There were actually—the 540,000 is a daily count. The transactions involved here were 2.9 million transactions.
Escudero: Involving how many depositors?
Consing: Involving 1.5 million depositors.
Escudero: At 2.9 million transactions, basic time and motion, 2.9 million transactions involving 1.5 million depositors, and the matter being addressed—in 37.
Consing: From point to point, 37 hours, we kept our electronic systems down for a total of 26 hours.
Escudero: Based on past experiences—the fact that they went out of their way by extending banking hours is up to them. Commendable, to say the least that they did that. But the response time, that’s my concern. Is that fast? Because by my standards that’s fast? But I don’t know by the standards of BSP, again, given the volume of transactions involved, and given the volume of depositors involved.
Anyone, Deputy Director Melchor Plabasan.
MELCHOR PLABASAN. So essentially, this type of incidents, I mean, this seldom happens. So it’s really difficult to benchmark. So even in other economies, these incidents happen and there are very limited information as to—I mean, the effort being undertaken. But we feel, I mean, given the gravity of the situation given, it’s fairly acceptable.
Escudero: Pending your official report?
Plabasan: Pending the official report because we still have to perform certain detailed audit procedures.
Escudero: And when will you wind up, coming up with the final report? And what, if any, should BPI still be doing in connections with this, moving forward?
Plabasan: Yes. Basically, as committed by the incoming governor, we have started the investigation but the first few days were really devoted to preliminary discussions— preliminary walk through of different processes.
Because at that time, we could not start with the detailed procedures because we do not want to interfere with the remediation efforts and then come Monday, it’s June 12. So essentially, we need a few more days….
Plabasan: … we need a few more days to conclude the investigation.
Escudero: Level of confidence with respect to what BPI has done to rectify the situation, as the regulators, Assistant Governor Fonacier or Director Tayag?
PIA BERNADETTE TAYAG: Good morning, Mr. Chair. Good morning, Senator Drilon:
Basically, the perspective that I will come from is really on consumer protection because there were consumers affected. And in terms of benchmark, what we want to ensure is that our supervised entities: number one, if there is an incident, communicate clearly to their customers; if there is service disruptions, what are the alternatives that are provided; and if there is any loss or any complaints, that they are adequately heard and there is mechanism for redress.
So, basically, in situations like these, those are the things that we look at. Without giving any—pending the final findings in this area as what was already presented without giving any additional information, I think our resource persons presented their advisories, the schedule of their advisories, the additional measures that they put in place. I’m just reiterating what has already been shown and juxtaposing that with the things that we require our supervised institutions to do.
In terms of process, we expect our supervised institutions to have a mechanism for customers to let them know if they have complaints or in this case, since it was represented that no funds were lost, in case there are other related complaints due to the service disruptions, to approach their bank.
If they are not satisfied, the Bangko Sentral puts in place also a consumer assistance mechanism where they can come to the bank, come to the BSP and we can further facilitate this discussion.
Escudero: Director Tayag, you are talking of a grievance procedure that the bank should set in place which we discussed earlier. The first line of defense, as Mr. Gotuaco said, would be through the branch.
Gotuaco. Yes, sir.
Escudero: So, sa sinumang maaaring apektado nito, akala nila apektado sila, o baka nakalimutan nila nag-withdraw, nagdeposit sila, they can go to any branch?
Gotuaco. Yes, sir.
Escudero: And to clarify, Director Tayag, there is no prescriptive period for this? Meaning, the depositor can be made aware of it, baka naalala niya o biglang nangailangan kaya tiningnan, there’s no prescriptive period?
Tayag: No prescriptive period because the customer might not be aware. But once they are made aware, we really encourage them to go to the bank kasi sila po ang makakasagot nang madalian kasi sila iyong may alam ng client information and transactions. Pero kung hindi po sila satisfied, puwede silang pumunta sa Bangko Sentral and we refer it back to the bank. Doon po mayroon nang turnaround time. We require the bank to respond to the BSP and to the client within seven banking days.
Escudero: Now, in this situation, when reversals are made, everything is borne by the bank, all the costs are borne by the bank. Iyong extra hours, hindi ba, hindi naman iyan pass-on. Am I correct, Mr. Consing?
Consing: That is correct. All costs are borne by the bank.
Escudero: So, reversals from within the bank’s affiliated firms are borne by the bank?
Consing: Everything within the bank is borne by the bank, all extra costs related to the extra hours, extra overtime, et cetera, is borne by the bank. The cost of communication is borne by the bank.
Escudero: Also borne by the bank?
Consing: Yes.
Escudero: Yes, Mr. Gotuaco?
Gotuaco. I concur, Mr. Chairman.
Escudero: So, it’s borne by the bank?
Consing: Yes, sir.
Escudero: Now, in theory, what if someone makes a claim? Sinabi niya, “Ay, nabawasan ako ng P20,000.” It’s difficult to prove from my end as the depositor. How will the bank now proceed to address such a claim by a depositor? Meaning, ano iyong pagbabanggain ninyong data to prove or disprove whatever claim a depositor may have with respect to a debit or a credit?
Gotuaco. Yes. That is—the fortunate circumstance here is that we know very precisely the nature of the mispostings. We know what cause balances to be too high or too low where ATM-related transactions had occurred between April 27 and May 2 that were posted on June 6.
In fact, sir, that’s why you could always go back to the main branch or your branch to transact on June 7 and June 8 because our branch personnel knew the proper balances of our clients as of the day prior and we were also able to determine on June 6 what were those errant transactions, those errant postings from April 27 to May 2 that were posted on that day. Kaya po, you could come and bank and do all your transactions at the branch on June 7 and June 8.
Escudero: So, kung nakalimutan man niya o may gustong magsamantala, what will you show him or her as proof na “Oo nga, nabawasan ka. Sorry, baka nakalimutan niyo na nag-withdraw kayo.”
Gotuaco. Yes. And so, even after the incident, if the client wanted to ascertain kung ano ho iyong tamang balance niya sa account niya, it’s very easy even after the incident to come to the branch, talk to your relationship manager, call our call center, and you will see that no money has been lost. Your account is intact.
Jocson: Your Honor, may I add to what Mr. Gotuaco mentioned?
We have had some cases, but we have our reconciliation process in the bank. When you withdraw from the ATM, there are snapshots. We have CCTVs, there are transaction logs. So, there are clients coming back and saying, “Hindi, sobra ang bawas mo.” We showed them the tape–kasi ho natural lang, April 27-28, makakalimutan mo. And then we showed them the transaction logs.
“At 6:00 p.m., that day, April 26, you withdrew this money and ito iyong screen captured, ito iyong CCTV.” So, we were able to address that. So, factually, the reconciliation process goes, we look at the logs, we look at, you know, what were the facts behind the claim, right, and pag legal and factually it’s proven na ganoon, we make good.
Escudero: How long before you can do that?
Jocson: Three days, Your Honor.
Escudero: Maximum of three days?
Jocson: Yes, Your Honor.
Escudero: Now, the CCTV footages with respect to ATM banks of BPI?
Jocson: ATM banks of BPI, Your Honor.
Escudero: ATM machines of BPI?
Jocson: ATM machines of BPI. But also, we have the transaction logs of when, let’s say, the switch—if they withdraw from another bank, there’s a switch message, that’s passed to us so we were able to confirm,
Escudero: But no more CCTV?
Jocson: No more CCTV, but at least we can show when it was withdrawn and there’s collaboration from the other bank, because it was transacted there.
Drilon: Just to emphasize, by personally appearing in any of the branches, a depositor could have immediately verified the accuracy of any entry or posting in his or in her account?
Jocson: Yes. I will also refer to Mr. Gotuaco if he need to add anything to this.
Gotuaco. Yes. That’s correct. And you could have done that on June 7 and June 8 and obviously on any given day, sir.
Escudero: Up to today.
Gotuaco. Yes, sir. Up to today.
Escudero: Before proceeding to BDO, any other additional statements, Mr. Consing?
Consing: Mr. Chairman, Your Honors, our focus has been on making sure we do right by our clients. Our focus has been 90% making sure that no one loses money. There had been double credits here but that has not been our focus, okay. We’ll sort out the double credits in good time, the amounts are small. Our focus has been almost exclusively on making sure that everyone is kept whole.
Escudero: The debits?
Consing: The debits, sir. So, if there’s anyone out there still outstanding, please come to us. We will fix this.
Escudero: Iyong credits?
Consing: We will do that in our good time.
Escudero: Paano pag na-withdraw na?
Consing: But the focus has been on the debit side.
Escudero: On the debit side?
Consing: Yes, sir.
Escudero: So far, has anyone filed an action or complaint against BPI from the point of view of any of the depositors of BPI as a result of this incident?
Consing: Not that I am aware of, Your Honor.
Escudero: Thank you, sir.
Drilon: Just curious. Would the Central Bank be aware of any similar incident abroad? Let’s say, in the most sophisticated, I mean, economies, do these things happen too or even in a more or the most sophisticated system?
Escudero: Deputy Director Plabasan, sir?
Plabasan: Yeah. Even in Europe and Singapore, these incidents really happen as previously mentioned.
Escudero: So, we are not unique, in a sense?
Plabasan: Yes, sir.
Escudero: In a bad way, we are not unique?. . .
Plabasan: Yes.
Drilon: And it is because human intervention could always cause something that is not intended to be. Is that correct?
Plabasan: Yes, sir.
Drilon: Thank you. But the important thing is the ability to respond immediately.
Plabasan: And to recover.
Drilon: And to recover.
Plabasan: Yes.
Skimming or cloning hit 7 of 3,700 BDO ATMs; less than 100 of 15,000 ATMs targeted
(Present at the BDO hearing were: Ricardo Martin, EVP and head, Information Technology Group, BDO Unibank, Inc.; Edwin Romualdo Reyes, EVP and head, Transaction Banking Group, BDO; Alvin Go, legal adviser, BDO; Peter Magdame, vice president, Transaction Banking Group, BDO; and Tomas Victor Mendoza, senior vice president, Transaction Banking Group, BDO)
Escudero: Moving forward, may I now move to the BDO—I’ll call it an incident or a situation. And who is the designated hitter on the part of BDO?
Mr. Reyes.
Sir, ulitin ko po yung sinabi ko kanina. Briefly, in layman’s terms, can you discuss to us what happened? What the current situation is? What has been done to rectify and correct? And what will be done to avoid the occurrence in the future of a similar incident?
You may proceed, Mr. Reyes.
Edwin Reyes, EVP and Head, Transaction Banking Group, BDO
First of all, we also would like to thank you for the opportunity to present BDO’s case and share with you in details on exactly what happened, what we have done to date and what we will do going forward to help improve the service that we provide to our clients.
I also like to introduce the team from BDO with me here, starting from Mr. Alvin Go, our legal adviser; we have Ricky Martin who is head of our Information Technology Group; Mr. Tovi Mendoza who heads the Product Management Team of which the ATM management is a part of; and then, of course, we have Peter Magdame also from Transaction Banking Group who, among many other functions, is in charge of the Fraud Management Unit of BDO.
As a general comment, as we talked about the data processing issue with BPI, we wish to state that any systems environment will always require some degree of human intervention. Intervention is required for a step requiring human judgement or a quality control, maintenance or upgrade that is best performed by a person, not a machine. Given this fact, human error is inevitable. Automation is only an accepted means to minimize human error and its impact on a business or an operation.
All systems environments have potential points of failure. The key to mitigating risk is to identify where these points of failure exist and have the requisite recovery procedures in place to kick in whenever failure occurs. All banks were dependent on technology heavily for the successful delivery of services to its clients.
As a result, banks are always careful and geared towards addressing and resolving problems quickly whenever there is a failure. Key functions include tight documentation and recordkeeping, timely balance reconciliations and the like. BDO adheres to the basic principles of quality control, ensuring a maker-checker arrangement for all transactions at any time. Going back to BDO’s case which concerns recent purported incidents of ATM skimming fraud, the bank assures the public that there is no cause for worry. ATM skimming fraud is common and affects ATMs and the cardholders of many banks.
Three fraud events
The recent events are related to three recent separate fraud events that have reasonably come to the bank’s attention and they affected seven ATMs. The seven ATMs are from three locations. The number of cases so far filed and we’ve reviewed and handled, there are about 95 cases and as a result, we have also disabled cards that we know have been compromised. The investigation of these cases are presently ongoing and the numbers we’ve provided are subject to change. In addition, the data refers to complaints and issues validly filed through the bank channels. As a general process we validate all complaints received and begin the process of rectification immediately once validated. To put this incident in its perspective, the recent fraud events involved seven ATMs out of a total of 3,700 for BDO and that is 0.2 percent of the ATM population.
Now, let me talk a little bit about skimming, Your Honor. Skimming is the unauthorized copying of the magnetic stripe information of ATM cards, the thin black stripe at the back of your cards which source the details of your card and is necessary for ATM transactions. Skimming is done through illegal devices that read or skim the magnetic stripe as transactions are being done. A second device is usually present to obtain the ATM PIN. In the Philippines, this is often a small pinhole camera that records the entry of a PIN during a transaction. The skimmed card details are then used to create fake cards which is paired to the recorded PIN and used to perform unauthorized withdrawals or purchases.
Skimming has been going on for a quite a number of years already. The magnetic stripe is a 50-year old technology and attempts to fraud it are as old. However, new technology makes skimming and in-camera devices very cheap, easy to produce and obtain.
Fraud Services also do a lot of research and development. New methods of attacks such as what we’ll show later. Deep insert skimmers which are very thin, small and hard to detect, arrived on a constant basis and often attempt to thwart whatever security measures we have implemented.
The three incidents above are related to the new type of skimming device, as I said, called the deep insert. Once this insert is in the machine, once resting there, they can read cards virtually undetected, and will not be visible through conventional anti-skimming protection modules. More importantly, they are also very easily installed and removed, with the process very similar to actions made by a client inserting an ATM card into the ATM machine.
This new insert device is coupled with newer upgraded versions of the pinhole camera with strong magnets that are capable of attaching to the PIN pad shield.
Just with the extent of compromised ATMs, the number is very small. In a recent study of the Philippine Banking Industry in a formal study in the first quarter of 2017, there were less than 100 ATMs hit out of the 15,000 ATMs nationwide, around 0.5 percent of total ATMs in the country. While there is admittedly a general upward trend in the numbers–we have seen that recent fraud attempts in May and June– generally ATMs are safe to transact in.
Now, Your Honor, what have we done? BDO continues to employ a number of measures. Some unique to us, of course, some—I am sure, the other banks are doing to secure our cardholders’ transaction.
Firstly, the bank will reimburse all clients affected by unauthorized withdrawals by said point of compromises (POCs) subject to existing investigation or reimbursement processes. It is worth noting that this requires the customers file their disputes through the proper bank channels. Social media post and activities will not be actionable from the bank’s side without a formal filing. And of course, card replacement will be free.
Secondly, for customers proactively blocked, these cardholders may proceed to their branch of account for a free card replacement.
Thirdly, on preventing deep insert skimmers, BDO is presently deploying an upgrade to its ATM machines.
We have the solution, that’s probably the one thing that maybe we would like to be careful in terms of sharing it, maybe later in a closed door session.
BDO upgrades 1,000 ATMs
But as of writing, over 1,000 of the bank’s ATMs including the seven point of compromises we’ve mentioned have been upgraded. The target implementation of the rest of the bank’s ATM fleet is the fourth quarter of this year. It takes time to do this in backlashing machine. And then additionally, the bank also deploys the following to protect its cardholders on what we call “this is business as usual” basis: No. 1, live real time fraud system to track our customer’s transactions and determine suspicious or off-pattern withdrawals and purchases.
Peter here is in-charge of that unit, fraud management unit.
Fully-staffed, 24/7 fraud team that does the monitoring of the system and also investigates cases that are filed with BDO. The fraud team can also anticipate a potential skimming attack and block cards pro-actively so that the fraudster will have no access to cardholder funds.
No. 3, physical and software security measures such as sensors to detect traditional skimming devices that are being attached on card readers, PIN pad covers to hide the PIN as customers transact; No. 4, security personnel both in the branches and in off-site ATM locations to check our ATMs for suspicious devices attached to the machine, as well as people who act so suspiciously around the machine; No. 5, regular ATM security reminders in traditional and social media with the focus on skimming; and lastly, the migration to EMV. We are not being talked about—BDO has been migrating customers to the newer EMV as early as 2016. But the bank is on track to its 2018 timeline for the migration.
That said, any bank card which uses magnetic stripe to transact—which covers all banks in the Philippines, by the way, and most of the world—is still vulnerable to this type of threat. Even this, BDO would like to reiterate that these purported skimming cases are isolated in nature and that the handling of customer issues is a business as usual process by the bank. The action points mentioned demonstrate BDO’s commitment to protect their customers and their transactions.
Incidents expected to happen
As a final message, I would like to say as well that having a broad network of ATMs provides benefits to banking clients— convenience, access options alternative to physical brick and mortar branches. Skimming and similar compromised incidents were expected to happen. It is nothing out of the ordinary, it’s BAU. That is the price we pay for providing the above benefits, conveniences to our clients.
We would like to reiterate what we already said to the public. There is no cause for worries. Skimming accidents can happen but are generally a small percent of total transactions. Banks, the BSP and the authorities are well-aware of these incidents and have all taken measures and continue to take measures to arrest said cases from happening—detected as they happen, and investigate the fraud events immediately.
And while we would like to reassure the public that all efforts are being done by banks to protect its cardholders and their transactions, cardholders can do their part by being vigilant when performing their transactions and reporting cases of suspicious devices and people around the ATM machines that they transact in.
Escudero: Sir, sisimplehan ko, skimming is theft. Gumamit ng teknolohiya para nakawan ng pera iyong isang depositor gamit ang ATM machine. Is that an accurate way of describing it?
Mr. Magdame, yes.
PETER MAGDAME: Yes, Your Honor.
Escudero: Now, if it’s theft, so nanakawan iyong depositor gamit itong scheme na ito, may paraan ba para malaman ng bangko kung ang ginamit iyong deep insert o ginamit niya iyong card niya? Ang pinupunto ko po ay ito, kung gumawa ng claim iyong isang depositor, can the bank actually find out and detect if it was the depositor himself using his card that made the transaction or the skimming device?
Anyone can answer, please.
TOMAS VICTOR MENDOZA: Your Honor, currently we do have what you call a point of compromise. This is when a lot of people complain, at the same time, of a particular transaction or particular point of time. So if, for example, a customer complained that I have not withdrawn in this particular ATM, we look back and trace all his transactions; and there is another customer from another bank, for example, complains of the same unauthorized withdrawal, we now have a matrix to pinpoint which is the point of compromise.
Which ATM was used as a point of compromise of the data of the cards, so that is how we look at it. At the same time, we have a system in BDO which has active monitoring of suspicious transactions.
For example, sir, if a particular customer does not withdraw abroad, it triggers already a flag to us na, “Oops, why is this suddenly, this customer has a transaction abroad?” And we will call the customer, “Have you done a transaction in the US?” So that is one of the things that we actually do.
Escudero: But, sir, that’s circumstantial. My question is, from the technological point of view, from the bank’s point of view, is there a way by which you can determine that it’s a skimmed device or a card that was used or the actual depositor’s card?
Because if I am the depositor that was skimmed, that is now circumstantial. What if I behaved extraordinarily for that day and made an additional out of character withdrawal or deposit? Meaning—may ganoon naman. How do you address that?
Magdame. Your Honor, skimming is actually a way of, in essence, cloning the card of a customer. So from the perspective of the bank, it would recognize that as a valid card. We would like to emphasize that the magstripe is actually a 50-year old technology. It is a bit dated. That this particular, shall we say, a peculiarity of the technology is something that Europay MasterCard Visa (EMV) can actually overcome.
Because for EMV, every transaction would be unique and every card, in essence, would be unique. So even if you attempt to clone it, the bank systems would recognize it as a skimmed card and will not be able to post your transaction.
Escudero: If it’s EMV already?
Magdame. Yes, sir. Yes.
Escudero: But right now, hindi pa?
Magdame. Sir, right now, we are actually doing the migration. The entire industry is moving to EMV already. And the bank has actually been complying to the said standard.
Drilon: With your permission, when can we complete the migration to the EMV system? When?
Magdame. The bank is on track with its 2018 target, sir.
Escudero: Correct me if am wrong, in previous discussions with the BSP, the BSP has already issued directives for all banks to migrate to EMV. The first deadline was supposedly January of 2017.
But given the volume, especially the bigger banks—I think the smaller banks have been able to comply—but the bigger banks are taking more time to migrate. Now, the deadline was extended to June 30 of 2018.
But in the meantime, Manong Frank, and this is where I am heading towards, there is also a regulation to the effect that if the bank has not yet migrated and given that this is old technology already, by BSP regulation, liabilities would fall on the bank for any loss that may be suffered—correct me, if I am wrong—by any depositor as a result of skimming, because this is theft, nanakawan iyong depositor. Kung nanakawan iyong depositor dahil doon, the depositor will not bear the loss. Is that a correct understanding?
Mendoza: Yes, sir. As earlier stated, after investigation and we found out that there is really a theft, we reimburse.
Escudero: At the cost of the bank.
Mendoza: At the cost of the bank.
Maybe, Your Honor, for you to appreciate how skimming is being done—
Escudero: May sample ka?
Mendoza: We are ready. So we brought a little show and tell of how technology has evolved.
So previously, when people try to steal cards, they have what you call skimming devices. This is one of the earlier generation skimming devices.
Escudero: Teka, tinuturuan ba natin yung nanonood kung paano gawin o–
Mendoza: Actually, sir, this is public information but at least we can show how is the impact. That’s it.
Escudero: Hindi naman tayo nagtuturo kung paano gawin ngayon, huh?
Mendoza: Hindi naman po.
Escudero: Okay.
EDWIN REYES. We will not show the solution.
Mendoza: Ipapakita ko kung paano–
Escudero: Okay.
Mendoza: [Demonstrating] So this is how a skimming device looks like. And if you remember an ATM, this is what you call an overlay. So what they do is that they add this. Pag nakakabit na, hindi mo na nakita.
Escudero: Ah, sinisingit sa labi.
Mendoza: Sa throat ng ATM. But you require two things. Your track data from the mag stripe, this one, and the pin. So paano nila kinukuha yung pin?
Previously what they do is what you call “shoulder surfing.” Yung may tao sa likod, sinisilip kung ano ang pin mo. Kaya ang ginawa natin, naglagay tayo ng mga mirror sa ATM. Mapapansin ninyo may mga salamin para makita mo kung may sumisilip.
Escudero: May rear-view mirror.
Mendoza: Plus the regulators have mandated to put what you call a “PIN pad shield” para nakatakip yung PIN. So iyong nasa makina po, ayan dito ho iyong PIN pad.
Escudero: Nakatago.
Mendoza: Oho. So ang ginawang solution ng fraudster, dahil may takip, gumawa po sila ng ganito. Ang tawag dito “PIN pad overlay.” So pinapatong ito sa taas ng PIN pad, sa loob nito, so maski na may takip, naka-capture pa rin nila yung data.
Escudero: Iyan yung camera?
Mendoza: Hindi po. Hindi pa po ito camera.
Escudero: Hindi pa iyan ang camera?
Mendoza: Ang ginagawa po nila dito, pag pumipindot po, kina-capture yung PINs.
Escudero: Pero bago niya mailagay iyan, kailangan niyang baklasin yung keypad noong ATM.
Mendoza: Actually po hindi, pinapatong lang, parang may glue lang po siya sa likod, heto po.
Escudero: Mukha siyang keypad.
Mendoza: Mukha siyang keypad po. So ang tawag dito is a PIN pad overlay. So nilalagay po nila doon.
Drilon: That will not be noticed by the user of the ATM?
Mendoza: Actually, sir, a lot of the users have been able to identify but unfortunately some did not. Kasi pag nagmamadali ka, madilim. So the enhancement of the overlays transaction came about. Because of the efforts of all the banks and the regulators to help us with the EMV—a lot of people now, what you do is they try to identify nga kung may overlay. So medyo nahuhuli na po yung overlay, napapansin na. So they started coming out with pinhole cameras because of the technology. So the original technologies that they were using was using either a camera on the machine or at the back of the machine and trying to get the PIN.
Escudero: Question. This one is hooked up to them or they have to retrieve this again in order to get the data?
Mendoza: Sir, you need to retrieve it.
Escudero: So nandito yung data?
Mendoza: Nandiyan po.
Escudero: Hindi ito online sa kanila?
Mendoza: Hindi po.
Escudero: Wala pa silang Internet.
Mendoza: Wala pa silang koneksiyon sa network, so talaga pong nakahiwalay po talaga siya.
Escudero: So yung data maiipon lang dito?
Mendoza: Opo.
Escudero: And they have to retrieve this in order to use it.
Mendoza: Yes, sir.
Drilon: Now, wouldn’t your CCTV be able to capture who would try to recover this?
Mendoza: Yes, sir, actually we do. Unfortunately, sometimes it’s after the fact.
Drilon: Obviously, after the fact kasi they will remove this after–
Mendoza: Kasi kung minsan, sir, mabilisan, ilang hours lang kinukuha na nila agad. So kukuha lang, magha-harvest sila ng kaunti—
Drilon: So did you say that there were eight incidents–
Escudero: Seven ATMs.
Mendoza: Seven ATMs, sir.
Escudero: In three locations.
Mendoza: Three locations recently.
Okay. Just to continue, sir. Well technology has evolved, medyo huli na yung mga tao, they started building what you call “pinhole cameras.” Ang pinhole camera, yung iba—this is just one example—was to use the same PIN pad cover, and if you notice, they have attached a camera here which is at the back. So what they do now is they capture the pins through the cameras.
(To be continued)
