The use of MS Chart Control and VB.NET for Statistical Purposes: Plotting Kaplan-Meier Estimate.

Introduction

Since the 17th century, the survival analysis appeared with the beginning of the development of actuarial science and demography. [1] Survival analysis, today being used extensively in clinical research, is not only confined to the analysis of actual survival, measuring the death rates, but has a much broader meaning including any “Event” an Investigator in a clinical trial might want to study. A research paper, published by Kaplan and Meier in 1958 and cited more than 33,000 times [2] presenting their “famous” survival curve estimator, led to a major advance in survival analysis.

Kaplan and Meier proposed a “continuous-time” version of the classical life table, the latter based on division of time into fixed intervals[1]. 

There are a lot of commercially available desktop software, that are used to calculate the Overall or Event-Free Survival and to plot the Kaplan-Meier Estimate; for example IBM SPSS ®  and MedCalc ®. Opensource statistical software like “R” are available, however, “standard” users will still need a specific training to be able to use the software efficiently. Web enabled data collection and analysis interface, though doable, is not an easy task to perform. 

We used VB.Net and MS Chart in a Web Application to Extract the data from a user-uploaded or online database, create life tables using the Kaplan-Meier Formula and plot the survival curve. The whole operation is performed through a user friendly web-based wizard.

The wizard starts by asking the user to choose a database table to analyze. Once selected, the user will match the “Date Start”, “Date End”, “Event” and “Basis for Stratification” with fields from his uploaded table.

The curve will then be plotted along with life table. The programmatically generated chart will describe survival data at specified time points through the tooltip property of the MS Chart.

Our tool provides a means to analyze survival data "real-time", the code can be manipulated to be compliant to the the study protocol statistical consideration section (for example setting the optimum follow up period). the produced life tables can be exported to be studied furthermore. Being a web-based application, no client set-up is needed. In contrast to other statistical software, no additional software is needed to be installed on the server. 

Background 

Let S(t) be the probability that a patient from a given study population will have a lifetime exceeding t [3,4].For a sample of size N from this population, let the observed times until death of the N sample members be

In the following example, ten patients are enrolled in a study, the “Duration” represents [Date End - Date Start] where Date Start is Date of Enrollment of the Study or Clinical Trial. Date End represent the Date of Death or Date of Event if “Death” or “Event” happens, respectively. If the patient is still alive or the specified “Event” did not happen, then Date End is the Last Follow-Up Date.

Corresponding to each ti is ni (the number "at risk" just prior to time ti), and di (the number of deaths or events at time ti ).

To apply this on the previous example, lets set the time points where “Event” = 1

First Time-point @ 3 months 

ni = 10

di = 1 

Second Time-point  @ 11 months

ni = 10 (initial) – 1 (patient died at timepoint 1) – 1 (patient alive at 9 months duration) = 8

di = 2 (two patient died @ duration of 11 months)

i.e. The 11 months’ survival for this example = 67.5 % 

Using the code 

The Project consists of two web-pages (Wizard and Plot) in addition to a VB.Net code file (SurvivalCurve.vb) placed in the "App_Code" folder. The following code is documented in-line. 

The Wizard Page 

The following are the ASP.Net controls needed to construct the wizard 

<asp:Label ID="Label1" runat="server" Text="Label"> </asp:Label> 
<asp:Label ID="Label2" runat="server" Text="Label"> </asp:Label> 
<asp:DropDownList ID="myList" runat="server"> </asp:DropDownList>
<asp:Button ID="Button3" runat="server" Text="Select Table" />
<asp:TextBox ID="choosetbl" runat="server">Table / Query Name</asp:TextBox>
<asp:Button ID="Button2" runat="server" Text="Cancel" />
<asp:Button ID="Button1" runat="server" Text="Next" /> 

 And the following is the code-behind file 

 Imports System.Data
Imports System.Data.OleDb
Partial Class DynamicKaplan
    Inherits System.Web.UI.Page
    Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
        If Session("ChosenTable") = "" Then InitMyDB()
    End Sub
    Private Sub InitMyDB()
        Dim myDB As New OleDbConnection
        Try
            myDB = New OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + Session("CurrentDB") + ";Persist Security Info=True")
            myDB.Open()
        Catch
            Threading.Thread.Sleep(1)
            Response.Redirect("Default.aspx")
        End Try
        'Retrieve the name of the current database
        Dim tmpString = myDB.DataSource.ToString
        Dim tmpInd = tmpString.LastIndexOf("|")
        tmpString = Right(tmpString, Len(tmpString) - (tmpInd + 1))
        Label1.Text = "The currently used Database is " & tmpString
        Label2.Text = "Please select a table or query from the database to estimate Kaplan Meier curves from: "
        'Retrieve the tables & queries schema
        Dim mytables = myDB.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, New Object() {})
        myDB.Close()
        'Iterate through the schema & fill the names to the dynamically created list
        '  myList.Items.Clear()
        
        For i = 1 To mytables.Rows.Count
            'to clean the list from annoying system tables
            If mytables.Rows(i - 1).Item(3).ToString = "VIEW" Then
                'Item(2 is the third column containing table name from the schema)
                myList.Items.Add(mytables.Rows(i - 1).Item(2).ToString)
            End If
        Next i
    End Sub
    Protected Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.Click
        If Session("ChosenTable") = "" Then
            'this is the first time click
            Session("ChosenTable") = choosetbl.Text
            Session("CurrentLoadState") = "Tables"
        End If
        LoadCases()
    End Sub
    Private Sub LoadCases()

        Dim CurrentLoadStateEx = Session("CurrentLoadState")
        Select Case CurrentLoadStateEx
            Case "Tables"
                'The user has pressed the "Next" Button for the first time, and chosen a table. It's time to fill the list with the available columns in the chosen table.
                InitSurvivalStatus()
                Session("CurrentLoadState") = "SurvivalStatus"
            Case "SurvivalStatus"
                Session("SurvivalStatusColumn") = myList.SelectedItem.Text
                Label1.Text = Label1.Text & ", 'EVENT STATUS' Column is '" & Session("SurvivalStatusColumn") & "' "
                Label2.Text = "Please select the field of 'REGISTRATION DATE' (The date of first contact) : "
                myList.Items.Remove(myList.SelectedItem)
                Session("CurrentLoadState") = "RegDate"
            Case "RegDate"
                Session("RegDateColumn") = myList.SelectedItem.Text
                Label1.Text = Label1.Text & "& the 'REGISTRATION DATE' Column is '" & Session("RegDateColumn") & "' "
                Label2.Text = "Please select the field of 'LAST CONTACT DATE' : "
                myList.Items.Remove(myList.SelectedItem)
                Session("CurrentLoadState") = "LastContact"
            Case "LastContact"
                Session("LastContactColumn") = myList.SelectedItem.Text
                Label1.Text = Label1.Text & "& the 'LAST CONTACT DATE' Column is '" & Session("LastContactColumn") & "'"
                Label2.Text = "Please select the field of 'STRATIFICATION' : "
                myList.Items.Remove(myList.SelectedItem)
                myList.Items.Insert(0, "None")
                Session("CurrentLoadState") = "Strata"
            Case "Strata"
                Session("Strata") = myList.SelectedItem.Text
                If Session("Strata") <> "None" Then
                    CheckStrata()
                Else
                    Session("StrataCount") = 1
                End If
                Response.Redirect("KaplanAccess.aspx")
        End Select
    End Sub
    Private Sub InitSurvivalStatus()
        Dim myDB = New OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & Session("CurrentDB") & ";Persist Security Info=True")
        myDB.Open()
        Label1.Text = "The currently chosen table or view is '" & Session("ChosenTable") & "'"
        Label2.Text = "Please select the field of 'SURVIVAL STATUS' where 1 means occurence of the event while 0 means no event : "
        'Retrieve the tables & queries schema
        Dim SqlString = "SELECT * FROM [" & Session("ChosenTable") & "];"
        Dim myDBCommand = New OleDbCommand(SqlString, myDB)
        Dim myReader As IDataReader = myDBCommand.ExecuteReader()
        Dim myColumns = myReader.GetSchemaTable
        myDB.Close()
        myList.Items.Clear()
        For I = 1 To myColumns.Rows.Count
            myList.Items.Add(myColumns.Rows(I - 1).Item(0).ToString)
        Next I
    End Sub
    Private Sub CheckStrata()
        Dim myDB = New OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + Session("CurrentDB") + ";Persist Security Info=True")
        myDB.Open()
        Dim SqlString = "SELECT DISTINCT [" + Session("Strata") + "] FROM [" + Session("ChosenTable") + "] WHERE ([" + Session("Strata") + "] Is Not Null);"
        Dim myDBCommand = New OleDbCommand(SqlString, myDB)
        Dim myReader As IDataReader = myDBCommand.ExecuteReader()
        Dim StrataCount As Integer
        Dim StrataArray As New ArrayList
        Do While myReader.Read
            StrataArray.Add(myReader.GetValue(0))
            StrataCount = StrataCount + 1
        Loop
        Session("StrataCount") = StrataCount
        Session("StrataArray") = StrataArray
    End Sub
    Protected Sub Button2_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button2.Click
        Response.Redirect("default.aspx")
    End Sub
    Protected Sub Button3_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button3.Click
        choosetbl.Text = myList.SelectedItem.Text
    End Sub
<pre lang="vb.net">End Class  

 The Plotting Page 

Here we add MS Chart as follows

<asp:Chart ID="Chart1" runat="server" Height="600px" Width="742px">
<pre lang="aspnet">     <ChartAreas>
            <asp:ChartArea Name="ChartArea1">
             </asp:ChartArea>
     </ChartAreas>
</asp:Chart> 

 and the code behind file

 Imports System.Web.UI.DataVisualization.Charting
Imports Microsoft.VisualBasic
Imports System.Data
Imports System.Data.OleDb
Imports System.Math

Partial Class KaplanAccess
    Inherits System.Web.UI.Page

    Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

        LoadKaplanData()
        'Chart visual settings
        Chart1.ChartAreas(0).AxisY.Maximum = 1
        Chart1.ChartAreas(0).AxisY.Interval = 0.1
        Chart1.ChartAreas(0).AxisX.MajorGrid.Enabled = False
        Chart1.ChartAreas(0).AxisY.MajorGrid.Enabled = False
        Chart1.ChartAreas(0).AxisX.Title = "Duration of Survival in months"
        Chart1.ChartAreas(0).AxisY.Title = "Cumulative Survival Propabaility"
        Dim myTitle As New Title
       
        myTitle.Text = "Kaplan Meier Survival Curve(s) for total of " & Session("TotalCasesCount").ToString & " cases"
        Chart1.Titles.Add(myTitle)

    End Sub
    Private Sub LoadKaplanData()
        'Get all settings from the session
        Dim CurrentDB = Session("CurrentDB")
        Dim ChosenTable = Session("ChosenTable")
        Dim SurvivalStatusColumn = Session("SurvivalStatusColumn")
        Dim RegDateColumn = Session("RegDateColumn")
        Dim LastContactColumn = Session("LastContactColumn")
        Dim StrataColumn = Session("Strata")
        Dim StrataCount = CInt(Session("StrataCount"))
        Dim StrataArray = CType(Session("StrataArray"), ArrayList)
        'Open the Database
        Dim myDB = New OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & CurrentDB & ";Persist Security Info=True")
        myDB.Open()
        'Base of the SQL String, Note that you can manipulate the DateDiff to add the optimum followup period recommended by the study protocol statistical consideration
        Dim SQLString = "SELECT [" & RegDateColumn & "], [" & LastContactColumn & "], [" & SurvivalStatusColumn & "] , DateDiff('m',[" & RegDateColumn & "],[" & LastContactColumn & "]) AS survivalduration" _
                        & " FROM([" & ChosenTable _
                        & "]) WHERE ((([" & RegDateColumn & "]) Is Not Null) AND (([" & LastContactColumn & "]) Is Not Null) AND (([" & SurvivalStatusColumn & "]) Is Not Null)) and ([" & RegDateColumn & "] < [" & LastContactColumn & "])"
        'Do this as many times as there is strata
        For I = 1 To StrataCount
            Dim FinalSQLString, SQLString2, Stratum As String
            SQLString2 = SQLString
            Stratum = "All cases"
            'Append strata selection criteria if there is any strata
            If StrataCount > 1 Then
                If IsNumeric(StrataArray.Item(I - 1)) Then
                    SQLString2 = SQLString & "and ([" & StrataColumn & "]=" & StrataArray.Item(I - 1).ToString & ")"
                Else
                    SQLString2 = SQLString & "and ([" & StrataColumn & "]='" & StrataArray.Item(I - 1).ToString & "')"
                End If
                Stratum = StrataArray.Item(I - 1).ToString
            End If
            'Formulate the final SQL String
            FinalSQLString = SQLString2 & "ORDER BY DateDiff('m',[" & RegDateColumn & "],[" & LastContactColumn & "]), " & SurvivalStatusColumn & " DESC;"
            Dim myDBCommand = New OleDbCommand(FinalSQLString, myDB)
            'Generate a temporary reader to get the number of cases
            Dim myReader As IDataReader = myDBCommand.ExecuteReader()
            Dim CasesCount As Integer = 0
            Do While myReader.Read
                CasesCount = CasesCount + 1
            Loop
            myReader.Close()
            'Generate the main reader
            myReader = myDBCommand.ExecuteReader()
            'Call the function which performs calculations
            Session("TotalCasesCount") = CInt(Session("TotalCasesCount")) + Kaplan.KaplanTables(Me, myReader, CasesCount, Stratum, Chart1)
            myReader.Close()
        Next I
        myDB.Close()
    End Sub
End Class  

 SurvivalCurve.vb 

Imports Microsoft.VisualBasic
Imports System.Web.UI.DataVisualization.Charting
Imports System.Data
Imports System.Data.OleDb
Imports System.Math
Public Class Kaplan
    Public Shared Function KaplanTables(ByRef myPage As Page, ByRef myReader As IDataReader, ByVal CasesCount As Integer, ByVal Stratum As String, ByRef myChart As Chart) As Int32
        'This function will take two variables, an OLE Database Connection with a table named "Query1", _
        'This query should contain the Survival status as the third column, and the survival duration as the fourth colum,
        'the other variable passed to this function should be a gridview to produce the lifetables in.
        'The function will bind the generated lifetables to the gridview & return the total number of processed cases as Int32


        'Start generating the life tables
        Dim myTable As New DataTable
        myTable.Columns.Add("Survial Status")
        myTable.Columns.Add("Remaining Patients")
        myTable.Columns.Add("Survial Duration")
        myTable.Columns.Add("Survial Propability")
        myTable.Columns.Add("Cumulative Survial Propability")
        Dim myFirstRow As DataRow = myTable.NewRow
        myFirstRow.Item(0) = 1
        myFirstRow.Item(1) = CasesCount
        myFirstRow.Item(2) = 0
        myFirstRow.Item(3) = 1
        myFirstRow.Item(4) = 1
        myTable.Rows.Add(myFirstRow)
        Dim Ptnseq = CasesCount
        For I = 1 To CasesCount
            Dim myRow As DataRow = myTable.NewRow
            'Get only one record  from KaplanTable
            Dim Kaplantmp = myReader.Read
            Ptnseq = Ptnseq - 1
            myRow.Item(0) = myReader.GetValue(2)
            myRow.Item(1) = Ptnseq 'Sets the total number of remaining patients
            myRow.Item(2) = myReader.GetValue(3)
            If myRow.Item(0) = 0 Then
                myRow.Item(3) = myTable.Rows(I - 1).Item(3)
                myRow.Item(4) = myTable.Rows(I - 1).Item(4)
            ElseIf myRow.Item(0) = 1 Then
                myRow.Item(3) = myRow.Item(1) / myTable.Rows(I - 1).Item(1)
                myRow.Item(4) = myRow.Item(3) * myTable.Rows(I - 1).Item(4)
            End If
            myTable.Rows.Add(myRow)
        Next I
        'Finished generating the lifetables, bind it to a grid
        Dim myGrid As New GridView 'Create a new dynamic Grid
        Dim myLabel As New Label 'Create a new dynamic label for this grid
        myPage.Form.Controls.Add(myLabel) 'add the label, then
        myPage.Form.Controls.Add(myGrid) 'add the grid
        myGrid.DataSource = myTable 'Bind the grid to the calculated lifetables
        myGrid.DataBind()
        DrawKaplanCurve(myTable, myChart, Stratum)
        myLabel.Text = "Current Stratum is: " & Stratum & "<br/>" & "Total Number of cases is: " & (myTable.Rows.Count - 1).ToString & " Cases" '(if  strata)
        Return myTable.Rows.Count - 1
    End Function
    Public Shared Sub DrawKaplanCurve(ByVal myTable As DataTable, ByVal myChart As Chart, ByVal Stratum As String)
        Dim KaplanSeries As New Series
        KaplanSeries.ChartType = SeriesChartType.StepLine
        KaplanSeries.Name = Stratum
        Dim CensoredSeries As New Series
        CensoredSeries.ChartType = SeriesChartType.Stock
        CensoredSeries.Name = "Censored " & Stratum
        For I = 1 To myTable.Rows.Count - 1
            Dim myPoint As New DataPoint
            Dim xval As Double = myTable.Rows(I).Item(2)
            Dim yval As Double = myTable.Rows(I).Item(4)
            myPoint.SetValueXY(xval, yval)

            ' If alive case, then add to censored data
            If myTable.Rows(I).Item(0) = 0 Then
                Dim CensoredPoint As New DataPoint
                CensoredPoint.SetValueXY(myPoint.XValue, yval - 0.01, yval + 0.01)
                CensoredPoint.ToolTip = "Censored Case Number " & myTable.Rows(I).Item(1).ToString & vbNewLine & "Survival Duration = " & myTable.Rows(I).Item(2).ToString & " months" & vbNewLine & "Cumulative Survival Propability = " & Round(yval * 100, 2).ToString & "%"
                CensoredPoint.Color = myPoint.Color
                If I <> myTable.Rows.Count - 1 Then CensoredSeries.Points.Add(CensoredPoint) 'add all except the last point because it shouldn't be censored
            End If
            
            If I = myTable.Rows.Count - 1 Then myPoint.Label = Round(yval * 100, 2).ToString & "%"
            KaplanSeries.Points.Add(myPoint)
        Next
        myChart.Series.Add(KaplanSeries)
        myChart.Series.Add(CensoredSeries)
       
        Dim myLegend As New Legend
        myLegend.TitleForeColor = myChart.Series(myChart.Series.Count - 1).Color
        myChart.Legends.Add(myLegend)
    End Sub
End Class 

References 

1. History of applications of martingales in survival analysis. Odd O.AALEN, PerKragh ANDERSEN , Ørnulf BORGAN, Richard D. GILL and Niels KEIDING. June 2009, Electronic Journal for History of Propability and Statistics, Vol. 5.

2. Non-parametric estimation from incomplete observations. Kaplan, E. L. and Meier. 1958, Journal of the American Statistical Association, pp. P.53, 457–481 and 562–563.

3. community, wiki. Kaplan–Meier estimator. wikipedia.org. [Online] 2012. 

4. Dunn, Steve. Survival Curves: Accrual and The Kaplan-Meier Estimate. cancerguide.org. [Online] 2002. http://www.cancerguide.org/scurve_km.html.